ML Infrastructure Engineer: Training & Inference

Physical Superintelligence; Boston (MA); Hybrid; USD 140,000 - 210,000; Full time

Job summary

Physical Superintelligence in Boston seeks a Member of Technical Staff, ML Engineer to build and run training and inference systems at scale, enabling researchers to focus on science rather than plumbing.

You will own distributed training, multi-GPU inference, and model-serving stacks (vLLM, SGLang, Triton), collaborate with engineers on observability and reliability, and ship with hands-on coding from day one.

Qualifications

• Three or more years building and operating ML training or inference infrastructure in production, at a company that trains or serves models at meaningful scale.
• Hands-on experience with distributed training (multi-GPU or multi-node, using PyTorch, Ray, or comparable) and model-serving systems (vLLM, SGLang, Triton, or comparable).
• Strong software engineering fundamentals. You can build a service that other engineers and researchers depend on every day, not a script that worked once.
• Enough ML fluency to work productively with AI researchers: you understand training loops, reward signals, and inference-time behavior well enough to debug them, even without designing the algorithms yourself.

Responsibilities

• Own the training and inference infrastructure that Core AI depends on: distributed training jobs, GPU scheduling, and model-serving systems (vLLM, SGLang, or comparable) for both proprietary models and self-hosted inference.
• Build the tools and abstractions AI researchers use to launch training runs, iterate on inference providers, and route workloads across models, so a researcher’s time goes into the science instead of the plumbing.
• Partner with Engineering on the shared platform: capacity planning, observability, and reliability for GPU and inference infrastructure, so training and serving hold up to the same production bar as everything else we ship.
• Debug and harden the training and inference stack under real load. Egress failures, stalled retries, and routing edge cases are your problem to close, not someone else’s ticket.
• Stay hands-on. You write the code, not just the design doc, and you are the first call when a training job stalls or an inference path breaks.

Skills

Distributed training

Multi-GPU / Multi-node

PyTorch

Ray

Model serving

Software engineering fundamentals

Collaboration with researchers

Tools

vLLM

SGLang

Triton

CUDA

Terraform

AWS

GCP

Job description

Physical Superintelligence in Boston seeks a Member of Technical Staff, ML Engineer to build and run training and inference systems at scale, enabling researchers to focus on science rather than plumbing.

You will own distributed training, multi-GPU inference, and model-serving stacks (vLLM, SGLang, Triton), collaborate with engineers on observability and reliability, and ship with hands-on coding from day one.