ML Infra Engineer: Scale Pipelines, GPUs & Platforms (Equity)

Alexander Chapman; New York (NY); On-site; USD 120,000 - 160,000; Full time

Benefits offered by this job

Equity

Health insurance

Dental & Vision

Job summary

Alexander Chapman is seeking a Mid-Level ML Infrastructure Engineer to build and scale the platforms and tooling that our data scientists rely on to train, deploy, and monitor models. You’ll focus on making systems around them fast, reliable, and easy to use.

Design and maintain training and inference infrastructure, build internal tooling for experiment tracking, feature stores, and model versioning, and optimize model serving for latency and cost at scale.

Qualifications

• 2–5 years of experience in infrastructure, platform, or backend engineering, ideally supporting ML workloads.
• Strong proficiency in Python and/or Go.
• Experience with containerization and orchestration (Docker, Kubernetes).
• Familiarity with ML-specific tooling: MLflow, Kubeflow, Ray, SageMaker, Vertex AI, or similar.
• Experience with cloud infrastructure (AWS, GCP, or Azure) and infrastructure-as-code (Terraform, Pulumi).
• Understanding of distributed systems and data pipeline design.
• Comfort working with GPUs and understanding of training/serving performance trade-offs.
• Strong communication skills and ability to work cross-functionally with ML practitioners.

Responsibilities

• Design and maintain training and inference infrastructure (pipelines, orchestration, compute scheduling).
• Build internal tooling for experiment tracking, feature stores, and model versioning.
• Optimize model serving for latency, throughput, and cost at scale.
• Set up and maintain CI/CD pipelines for ML workflows.
• Manage GPU/compute resource allocation and cluster infrastructure (e.g., Kubernetes, Ray, Slurm).
• Implement monitoring and alerting for model performance, data drift, and system health.
• Partner with ML engineers and data scientists to understand their workflows and remove friction.
• Contribute to platform architecture decisions as the ML org scales.

Skills

Python

Go

Docker

Kubernetes

MLflow

Kubeflow

Ray

SageMaker

Vertex AI

Terraform

Pulumi

Tools

MLflow

Kubeflow

Ray

SageMaker

Vertex AI

Docker

Kubernetes

Terraform

Pulumi

Triton

TorchServe

vLLM

DVC

Feast

LakeFS

Job description

Alexander Chapman is seeking a Mid-Level ML Infrastructure Engineer to build and scale the platforms and tooling that our data scientists rely on to train, deploy, and monitor models. You’ll focus on making systems around them fast, reliable, and easy to use.

Design and maintain training and inference infrastructure, build internal tooling for experiment tracking, feature stores, and model versioning, and optimize model serving for latency and cost at scale.