Senior Software Developer – Machine Learning Platform

Coveo; Montreal (administrative region); On-site; CAD 110,000 - 150,000; Full time

Job summary

Coveo in Montreal, Canada is seeking a Backend Developer to help build scalable AI-powered search and inference services. You will contribute to the development lifecycle, implement reliable backend systems in a cloud environment, and collaborate with scientists, data engineers and developers to deploy agentic workflows.

Ideal candidates have 5+ years in backend development, strong knowledge of distributed systems, and comfort with learning new tools and best practices.

Qualifications

• 5+ years of backend development in a cloud environment with Python or Java/Spring.
• Solid understanding of distributed systems and scalable design.
• Strong problem-solving and debugging skills with guidance when needed.
• Comfortable learning new tools, technologies, and best practices.

Responsibilities

• Contribute to the development lifecycle, including coding, testing and deployment.
• Analyze and improve observability, reliability, and usability of backend services.
• Maintain services handling millions of requests and LLM calls daily.
• Collaborate with data scientists, data engineers, and developers to productionize agentic apps.
• Help evolve the platform to support growth of agentic workflows for Coveo users.
• Suggest process improvements and leverage AI coding tools to aid development.

Skills

Backend development

Distributed systems

Debugging

Cloud environments

Python

Java/Spring

AWS

Tools

Terraform

AWS Bedrock

OpenSearch

Vector databases

LLM inference gateways

Job description

Build the backbone of AI at scale

Want to see your work power millions of machine learning predictions every day? Join our Machine Learning Platform team and help bring models, including Large Language Models (LLMs), into production in a reliable and scalable way.

Build the backbone of AI at scale

Want to see your work power millions of machine learning predictions every day? Join our Machine Learning Platform team and help bring models, including Large Language Models (LLMs), into production in a reliable and scalable way.

You’ll contribute to a platform that trains thousands of models and serves hundreds of millions of queries daily, while growing your expertise in distributed systems, machine learning infrastructure and agentic workflows.

As a Backend Developer, you will:

• Contribute to the development lifecycle, including coding, testing, and deployment
• Analyze and help improve the ease of use, observability and reliability of our AI platform
• Participate in maintaining services handling millions of requests and LLM calls per day
• Collaborate with applied scientists, data engineers, and developers to integrate agentic applications into production
• Contribute to evolving our AI platform to support the growth and scale of agentic workflows for Coveo users
• Suggest improvements to existing processes and use AI coding tools to support your development workflow

Here is what will qualify you for the role:

• 5+ years of experience in backend development in a cloud environment (Python or Java/Spring preferred, AWS an asset)
• Good understanding of distributed systems and scalable application design
• Ability to solve problems and debug systems with guidance when needed
• Comfortable learning new tools, technologies, and best practices

What will make you stand out:

• Familiarity with Terraform, AWS Bedrock, OpenSearch serverless or other vector databases, LiteLLM or other LLM inference gateways
• Exposure to machine learning workflows or agentic workflows
• Interest in scalable backend services, large-scale realtime systems, agentic workflows and applications or realtime inference at scale

You don’t need to check every single box; passion goes a long way and we appreciate that skillsets are transferable.

Join the Coveolife!

We encourage all qualified candidates to apply regardless of, for example, age, gender, disability, gaps in CV, national or ethnic background.

This job description was written by humans, assisted by AI. We may leverage technology in our hiring process to help us see the person behind the resume.

Coveo is committed to providing accessible employment practices. If you require accommodation due to a disability at any point during the recruitment process, please contact [e-mail] to discuss your needs.