ML Infrastructure Engineer — Build Scale Training & Serving

Harrison Clarke; San Francisco (CA); On-site; USD 180,000 - 280,000; Full time

Job summary

Harrison Clarke partners with a well-funded frontier AI lab to pursue a Machine Learning Infrastructure Engineer who will design and build the core infrastructure powering next-generation large-scale model training and deployment.

You will own the stack for training orchestration, distributed compute, and model serving, collaborating with researchers to deliver reliable, scalable platforms that accelerate AI research and production.

Qualifications

• 5+ years of software engineering with ML infra focus.
• Experience with ML tooling and distributed systems.
• Expertise in Python and GPU programming.

Responsibilities

• Design foundational ML infrastructure for training orchestration, distributed compute frameworks, GPU/TPU cluster management, and experiment tracking systems.
• Optimize large-scale training pipelines for throughput, fault tolerance, checkpointing, and resource utilization across multi-node, multi-accelerator environments.
• Develop and maintain model serving infrastructure with low-latency inference, packaging, deployment pipelines, and autoscaling.
• Build internal platforms and tooling for self-service compute, data pipelines, and reproducible experiment workflows.
• Collaborate closely with research teams to translate bleeding-edge ML requirements into reliable, scalable systems.
• Drive infrastructure reliability with monitoring, observability, capacity planning, and incident response for mission-critical ML workloads.

Skills

Python

Distributed systems

ML infrastructure

GPUs/accelerators

Kubernetes

Large-scale training

Experiment tracking

Tools

PyTorch

JAX

DeepSpeed

Megatron

CUDA

NCCL

InfiniBand

Job description

Harrison Clarke partners with a well-funded frontier AI lab to pursue a Machine Learning Infrastructure Engineer who will design and build the core infrastructure powering next-generation large-scale model training and deployment.

You will own the stack for training orchestration, distributed compute, and model serving, collaborating with researchers to deliver reliable, scalable platforms that accelerate AI research and production.