Senior LLMOps

Marler & Associates Search; Quebec; On-site; CAD 120,000 - 180,000; Full time

Benefits offered by this job

Stock options

Flexible remote work

Focus on long-term career growth

Job summary

A leading HR Tech company is seeking a Senior LLMOps to design and manage the infrastructure for a conversational AI platform. This remote role focuses on ensuring ML systems are scalable and efficient, with responsibilities including collaboration with cross-functional teams for infrastructure needs, compliance adherence, and cloud optimization. Candidates should have extensive experience in LLMOps, cloud engineering, and be excited about transforming hiring through AI. Competitive salary and stock options offered.

Qualifications

• 2+ years of experience in LLMOps, preferably in a SaaS or AI startup.
• 5+ years of experience in cloud engineering.
• Proven experience with deploying and maintaining LLMs.
• Experience with deploying and maintaining LLMs and conversational AI frameworks (e.g. Gemini API, OpenAI API).
• Experience integrating LLMs into rule-based or orchestration pipelines.
• Familiarity with monitoring and observability tools (Prometheus, Grafana, ELK, Datadog, etc.).
• Strong problem-solving, analytical, and communication skills

Responsibilities

• Design and maintain infrastructure for LLM-powered simulations.
• Collaborate with ML Engineers and Backend Developers for infrastructure needs.
• Automate training, evaluation, and deployment workflows for prompts, agents, and scoring components.
• Collaborate with ML engineers, backend developers, and product managers to define infrastructure needs and system architecture.
• Ensure GDPR and SOC 2 compliance through data protection, access controls, and audit-ready logging.
• Monitor, log, and report model and infrastructure performance; provide tamper-proof, detailed metadata for audits.

Skills

Cloud engineering

Strong problem-solving skills

Analytical skills

Communication skills

Kubernetes

Terraform

Helm

OpenAI API

Gemini API

Monitoring & observability

Tools

AWS

Docker

Kubernetes

Terraform

Helm

Prometheus

Grafana

ELK

Datadog

Job description

Senior LLMOps

Location: Remote (Eligible to work in Canada; Preference to residents of Quebec)
Type: Full-time
Start Date: ASAP
Team: R&D
Industry: HR Tech / Conversational AI / SaaS

Our client is redefining the talent acquisition game by designing conversational assessments that let candidates demonstrate their skills.

They have proven that technology can assess soft skills more accurately and fairly than people can. Their mission is to help hiring teams make excellent hiring choices; ethically, effortlessly, and confidently, through skills-first automation.

You’ll play a key role in advancing this mission while shaping the future of HR tech.

Your Mission

Our client is looking for a Senior LLMOps to design and manage the infrastructure powering their next-generation conversational AI platform. You’ll ensure their ML systems are robust, scalable, and efficient, enabling real-time, LLM-powered simulations that help global enterprises hire better, faster, and with confidence.

About the Project

Our client is building an AI-powered platform that simulates real conversations using LLMs to measure communication, empathy, and decision-making skills.

As an essential member of the project pod, you will be responsible for ensuring our ML infrastructure is scalable, robust, and efficient, from experimentation to deployment.

What You’ll Do

Core LLM Infrastructure & Deployment

• Design, build, and maintain infrastructure for LLM-powered simulations, via direct model serving (e.g., OpenAI, Gemini, Claude, local models) or orchestration frameworks like LangChain.
• Develop and manage deployment and integration pipelines for both ML and backend services.
• Automate training, evaluation, and deployment workflows for prompts, agents, and scoring components.

Collaboration & Optimization

• Collaborate closely with team members, including ML Engineers, Backend Developers, and Product Managers, to define infrastructure needs and system architecture.
• Work with the Legal and Compliance teams to ensure all LLMOps practices meet internal and external regulatory requirements, specifically for GDPR and SOC 2.
• Optimize cloud usage (AWS) for cost-efficiency and scale.
• Work with the cloud optimization team with tooling to identify overspending and paths to cost-efficiency.

GDPR & SOC 2 Compliance

• Implement data pseudonymization and anonymization techniques for all data used in LLM training, evaluation, and logging, particularly for any personally identifiable information (PII), to adhere to GDPR’s data minimization principle.
• Establish and enforce strict access controls (e.g., least privilege) across the LLM infrastructure and data stores to meet the Security and Confidentiality principles of SOC 2.

Monitoring, Logging, & Reporting

• Implement and manage monitoring, logging, and reporting systems for model and infrastructure performance.
• Provide a robust logging and monitoring mechanism that is tamper-proof and includes detailed metadata (e.g., source, time, model version) necessary for both operational debugging and compliance audits.

What Success Looks Like

• You share our clients passion for transforming hiring through AI.
• You engage in respectful, constructive collaboration and challenge ideas with data and empathy.
• You continually improve both our clients systems and yourself.
  You deliver a measurable impact on uptime, scalability, and cost efficiency.

Job Requirements

• 2+ years of experience in LLMOps, preferably in a SaaS or AI startup.
• 5+ years of experience in cloud engineering.
• Proven experience with cloud platforms (AWS), containerization (Docker, Kubernetes), and infrastructure-as-code tools (Terraform, Helm).
• Experience with deploying and maintaining LLMs and conversational AI frameworks (e.g. Gemini API, OpenAI API).
• Experience integrating LLMs into rule-based or orchestration pipelines.
• Familiarity with monitoring and observability tools (Prometheus, Grafana, ELK, Datadog, etc.).
• Strong problem-solving, analytical, and communication skills

Salary + benefits

Our client is offering a competitive salary, plus flexible 100% remote working, stock options, and a focus on nurturing your long-term career goals.

Why Join

• Shape a new AI-driven product from the ground up, with real-world impact in agent evaluation and conversational AI.
• Collaborate with an interdisciplinary, mission-driven team of ML engineers, linguists, and designers.
• 100% remote flexibility, stock options, and a focus on your long-term growth.
• Work in a diverse and inclusive environment where merit matters more than background.