Machine Learning Infrastructure Engineer

true-talent-solutions-partners; San Francisco (CA); Remote; USD 200,000 - 400,000; Full time

Benefits offered by this job

Remote-friendly

Competitive salary

Job summary

true-talent-solutions-partners is hiring an ML infrastructure engineer to design, deploy, and maintain distributed training and inference clusters for large-scale models. You will build scalable pipelines for petabyte-scale datasets and optimize performance across the ML lifecycle.

The role requires deep GPU expertise (CUDA/JAX), familiarity with FSDP/DeepSpeed, and cloud/container tech (GCP/AWS/Azure, Docker, Kubernetes). This remote-friendly position spans the US, with SF as a primary locale.

Qualifications

• 2–10 years of experience building large-scale ML infrastructure for pretrained or foundation models.
• Experience optimizing training, data, or inference workloads, including low-level GPU work with CUDA and JAX.
• Proficiency with distributed training frameworks such as FSDP or DeepSpeed.
• Experience with cloud platforms (GCP/AWS/Azure) and containers/orchestration (Docker/Kubernetes).
• Familiarity with scalable model serving, distributed task management, observability, and version control practices.
• Background in physical AI or AI for science and willingness to work across the ML lifecycle.

Responsibilities

• Design, deploy, and maintain distributed ML training and inference clusters.
• Build scalable end-to-end pipelines for petabyte-scale datasets and model training.
• Evaluate parallelization methods and numerical precision trade-offs at different model scales.
• Profile and debug low-level GPU operations to improve performance.
• Track relevant research and apply ideas to infrastructure and workflows.

Skills

Distributed training infrastructure

Model serving

Observability & monitoring

GPU performance tuning

Learning unfamiliar domains

Tools

CUDA

JAX

FSDP

DeepSpeed

Docker

Kubernetes

GCP

AWS

Azure

Job description

ABOUT THE ROLE

Join an AI for science team building large-scale models for physical systems. You will develop the infrastructure behind distributed training and inference, helping the team advance models that can predict outcomes and inform effective actions.

WHAT YOU'LL DO

• Design, deploy, and maintain distributed machine learning training and inference clusters.
• Build scalable end-to-end pipelines for petabyte-scale datasets and model training across the machine learning lifecycle.
• Evaluate parallelization methods and numerical precision trade-offs at different model scales.
• Profile and debug low-level GPU operations to improve performance.
• Track relevant research and apply promising ideas to infrastructure and model workflows.

WHAT WE'RE LOOKING FOR

• 2 to 10 years of experience building large-scale machine learning infrastructure for pretrained or foundation models, including compute, data, inference, and distributed training systems.
• Experience optimizing large-scale training, data, or inference workloads, including low-level GPU performance work with CUDA and JAX.
• Proficiency with distributed training frameworks such as FSDP or DeepSpeed.
• Experience with cloud platforms such as GCP, AWS, or Azure, and with containers and orchestration tools such as Docker and Kubernetes.
• Familiarity with scalable model serving, distributed task management, monitoring, logging, observability, and version control practices.
• Background in physical AI or AI for science, such as robotics, autonomous systems, or biology, and an interest in working across the machine learning lifecycle.
• Comfort working in a fast-paced environment and learning unfamiliar domains.

COMPENSATION & BENEFITS

Base salary range: $200,000 to $400,000 annually. Visa sponsorship is not available.

LOCATION Remote within the United States, with San Francisco, California as the primary location.