Remote MLOps Engineer: LLM Performance & Serving

Mercor; United States; Remote; USD 140,000 - 200,000; Full time

Job summary

Mercor is seeking a hands-on ML systems engineer for a fully remote position, building foundational AI model infrastructure across accelerators, serving large language models, and optimizing performance. The role emphasizes kernel programming, profiling, and distributed workloads within a world-class GenAI team.

You will collaborate with researchers and engineers to evaluate ML system tasks, provide feedback, and contribute to training data for frontier AI.

Qualifications

• 2+ years of hands-on professional experience in ML systems, ML infrastructure, model serving, or accelerator performance engineering.
• Experience writing or optimizing custom kernels (Pallas, Triton, CUDA, CUTLASS).
• Production experience with JAX and/or PyTorch.
• Ability to engage reliably for at least 40 hours/week during weekdays.
• Strong written communication and ability to explain complex technical decisions clearly.

Responsibilities

• Design tasks across kernels, profiling, debugging and inference serving, including adversarial problems that expose where frontier models fail on accelerator code.
• Guide research and engineering teams to close knowledge gaps and improve AI model performance on ML systems, compilers and training infrastructure topics.
• Evaluate ML systems tasks and solutions, and provide clear written technical feedback that stands up to expert review.
• Develop guidelines and detailed rubrics covering kernel optimization, profiler output interpretation, distributed systems reasoning, and serving throughput and latency trade-offs.
• Collaborate with other subject matter experts to keep training data consistent and accurate.

Skills

ML systems experience

Kernel programming

Profiling & tracing

Distributed debugging

LLM serving at scale

JAX/PyTorch production

Distributed training / compiler

HW accelerators (TPU/HW)

Technical communication

Tools

JAX

PyTorch

XLA

Kineto

Nsight

TensorRT-LLM

Job description

Mercor is seeking a hands-on ML systems engineer for a fully remote position, building foundational AI model infrastructure across accelerators, serving large language models, and optimizing performance. The role emphasizes kernel programming, profiling, and distributed workloads within a world-class GenAI team.

You will collaborate with researchers and engineers to evaluate ML system tasks, provide feedback, and contribute to training data for frontier AI.