Senior ML Infra Engineer: Build & Scale AI Systems

Goliath Partners Inc.; San Francisco (CA); On-site; USD 180,000 - 240,000; Full time

Job summary

Goliath Partners Inc. in San Francisco seeks an exceptional ML Infrastructure Engineer to own and scale production ML infrastructure.

You will design distributed training and high‑throughput inference systems, collaborating with research and platform teams to push performance and reliability at scale. You bring 5+ years in ML infra or distributed systems, strong Python and C++/Go, and hands-on experience with PyTorch/JAX, CUDA, and cloud GPU environments.

Qualifications

• 5+ years in ML infrastructure, distributed systems, or equivalent.
• Strong programming in Python and C++ or Go.
• Hands-on experience with PyTorch, JAX, or similar ML frameworks.
• Experience building scalable distributed training and inference infrastructure.
• Knowledge of Kubernetes, Docker, AWS/GCP, and GPU compute environments.
• Experience optimizing system performance, reliability, latency, and throughput.
• Proven ability to design and own production ML infrastructure at scale.

Responsibilities

• Design and own production ML infrastructure at scale.
• Build scalable distributed training and inference systems.
• Optimize performance, reliability, latency, and throughput.
• Collaborate with research and platform teams to advance AI workloads.

Skills

Python

C++

Go

PyTorch

JAX

Kubernetes

Docker

AWS

GCP

GPU compute environments

Education

Bachelor's degree in Computer Science/Computer Engineering/Mathematics

Master's degree in a related field

PhD in CS/ML/Distributed Systems (a plus)

Tools

CUDA

NCCL

Ray

DeepSpeed

Megatron-LM

Job description

Goliath Partners Inc. in San Francisco seeks an exceptional ML Infrastructure Engineer to own and scale production ML infrastructure.

You will design distributed training and high‑throughput inference systems, collaborating with research and platform teams to push performance and reliability at scale. You bring 5+ years in ML infra or distributed systems, strong Python and C++/Go, and hands-on experience with PyTorch/JAX, CUDA, and cloud GPU environments.