Remote MLOps Engineer: LLM Serving & GPU Profiling

24 Mag; New York (NY); Remote; USD 124,000 - 165,000; Full time

Job summary

24 MAG is seeking an experienced MLOps Engineer for a remote, full-time engagement focused on LLM systems, GPU kernels, profiling, and distributed debugging.

You will develop reference solutions, evaluate model outputs, and help set evaluation standards across GPU kernels, profiling, and high‑throughput serving. The role emphasizes production infrastructure experience over pure modelling.

Qualifications

• 2 years of hands-on professional experience in ML systems, ML infrastructure, model serving, or accelerator‑performance engineering
• Strong practical experience in at least one of GPU kernel programming, performance profiling, distributed debugging, or high‑throughput inference serving
• Production experience with JAX and/or PyTorch
• Familiarity with CUDA, Triton, Pallas, or comparable accelerator‑programming technologies
• Experience with profiling tools such as Kineto, torch.prof profiler, Nsight, XLA, or JAX profiler
• Experience debugging distributed or accelerator‑bound workloads
• Familiarity with vLLM, SGLang, TensorRT‑LLM, Ray Serve, KV cache, paged attention, or continuous batching
• Framework‑level experience with custom operators, FSDP, DDP, DeepSpeed, Megatron, compiler, or graph‑level work is highly valuable
• Familiarity with accelerators such as A100, H100, B200, or TPU
• Ability to reason precisely about throughput, latency, memory, and compute trade‑offs
• Demonstrable professional progression in ML infrastructure or systems engineering
• Strong written communication and ability to explain complex technical decisions clearly

Responsibilities

• Design GPU/accelerator workloads and kernel tasks.
• Develop CUDA, Triton, or Pallas‑based solutions.
• Evaluate kernel optimisation for correctness and efficiency.
• Analyse memory, compute, and hardware utilisation trade‑offs.
• Apply accelerator engineering judgement to model‑generated solutions.
• Create tasks for performance profiling and trace interpretation.
• Analyse outputs from Kineto, torch.profiler, Nsight, XLA, or JAX profilers.
• Identify bottlenecks across compute, memory, and scheduling.
• Evaluate throughput, latency, and utilisation characteristics.
• Produce clear analyses explaining performance behaviour.
• Design scenarios for distributed or accelerator‑bound ML workloads.
• Diagnose failures across training and inference infra.
• Evaluate FSDP, DDP, DeepSpeed, Megatron, and related systems.
• Review framework‑level and distributed‑system troubleshooting.
• Identify plausible but incorrect explanations or fixes.
• Develop tasks for high‑throughput LLM serving.
• Apply vLLM, SGLang, TensorRT‑LLM, Ray Serve.
• Evaluate KV‑cache, paged‑attention, and continuous batching.
• Analyse serving architectures for latency, memory, scalability.
• Review production‑grade inference deployment approaches.
• Evaluate MLOps and ML‑systems tasks and proposals.
• Provide precise written feedback for technical reviews.
• Develop rubrics and evaluation frameworks for systems work.
• Help research/engineering teams close knowledge gaps.
• Collaborate with SMEs to maintain data quality

Skills

ML systems engineering

GPU kernel programming

Performance profiling

Distributed debugging

High-throughput inference

Technical writing

Tools

CUDA

Triton

Pallas

Kineto

torch.profiler

Nsight

XLA

JAX

PyTorch

TensorRT-LLM

Ray Serve

vLLM

Job description

24 MAG is seeking an experienced MLOps Engineer for a remote, full-time engagement focused on LLM systems, GPU kernels, profiling, and distributed debugging.

You will develop reference solutions, evaluate model outputs, and help set evaluation standards across GPU kernels, profiling, and high‑throughput serving. The role emphasizes production infrastructure experience over pure modelling.