Machine Learning Engineering Manager - LLM Serving (Remote - US)

Jobgether; United States; Remote; USD 176,166 - 251,666; Full time

Benefits offered by this job

Competitive salary range

Comprehensive health insurance

Paid parental leave

401(k) retirement plan

Monthly meal allowance

Paid vacation days

Flexible remote/hybrid work options

Job summary

A leading company in AI solutions is seeking a Machine Learning Engineering Manager to lead a team in developing high-scale LLM serving infrastructure. The role involves managing projects and collaborating with ML researchers to optimize system performance. Ideal candidates have a strong background in software engineering and experience in managing teams. The position comes with competitive salary and comprehensive benefits, including flexible working options.

Qualifications

• 5+ years of experience in software or machine learning engineering.
• 2+ years managing engineering teams.
• Hands-on ML engineering background with production-quality systems.

Responsibilities

• Lead an engineering team to design and deploy LLM serving infrastructure.
• Collaborate closely with product and personalization teams.
• Develop observability and monitoring frameworks for ML systems.

Skills

Software or Machine Learning Engineering experience

Management experience

ML engineering hands-on experience

Technical expertise in LLM infrastructure

Leadership and communication skills

Tools

MLFlow

Job description

Machine Learning Engineering Manager - LLM Serving (Remote - US)

We are currently looking for a Machine Learning Engineering Manager - LLM Serving & Infrastructure in the United States.

This role offers the opportunity to lead and shape the development of high‑scale, low‑latency LLM serving infrastructure for advanced personalization and recommendation systems. You will manage a talented engineering team while collaborating closely with machine learning researchers, data scientists, and product teams to deliver robust, scalable, and cost‑efficient ML systems. The position focuses on deploying and maintaining LLMs across production environments, optimizing performance and reliability, and driving adoption of innovative AI‑powered features. This is a highly visible role where your technical leadership and strategic vision will directly impact millions of users, improving their experience through intelligent recommendations and interactions.

Accountabilities

• Lead a high‑performing engineering team to design, build, and deploy large‑scale LLM serving infrastructure.
• Drive the implementation of a unified serving layer for multiple LLM models, supporting batch, offline evaluation, and real‑time inference.
• Oversee the development and management of the Model Registry, including deployment, versioning, and monitoring of LLMs across production environments.
• Collaborate with internal product and personalization teams to integrate LLM‑powered features seamlessly into recommendation systems.
• Establish standardized technical interfaces, protocols, and best practices for efficient model deployment, scaling, and operational monitoring.
• Optimize serving infrastructure for latency, cost, and reliability, leveraging techniques like quantization, pruning, and efficient batching.
• Develop observability and monitoring frameworks, including dashboards, alerting, and SLA tracking for inference traffic and system health.

Requirements

• 5+ years of software or machine learning engineering experience, including 2+ years managing engineering teams.
• Hands‑on experience in ML engineering, building and scaling production‑quality ML systems and datasets.
• Strong technical expertise in LLM infrastructure, recommendation or personalization systems, and high‑volume, high‑velocity ML platforms.
• Proven ability to lead complex, multi‑partner projects with federated contribution models.
• Experience designing loosely coupled, scalable systems with clear separation of concerns and robust APIs.
• Familiarity with CI/CD pipelines, experiment tracking, and results visualization tools (e.g., MLFlow).
• Excellent leadership, communication, and collaboration skills, with the ability to influence peers and stakeholders.
• Pragmatic, results‑oriented mindset, balancing speed and production rigor.

Benefits

• Competitive salary range: $176,166 – $251,666 plus equity.
• Comprehensive health insurance coverage.
• Six months paid parental leave.
• 401(k) retirement plan.
• Monthly meal allowance.
• 23 paid vacation days, 13 flexible holidays, and paid sick leave.
• Flexible remote/hybrid work options with occasional in‑person meetings.
• Opportunity to work on cutting‑edge ML infrastructure impacting millions of users globally.