ML Systems Ops Engineer: Training Clusters & Inference Cloud

Zyphra Technologies Inc.

San Francisco (CA)

On-site

USD 180,000 - 230,000

Full time

14 days+

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Benefits offered by this job

Comprehensive medical, dental, vision
401(k) plan
Relocation and immigration support
In-office snacks and meals
Unlimited PTO and company holidays
In-person team in San Francisco

Job summary

Zyphra Technologies Inc. seeks a Machine Learning Systems Administrator/DevOps Engineer to maintain core infrastructure for model training and inference.

The role spans large training clusters and production inference services across heterogeneous accelerators, focusing on reliability, security, and scalable workflows. You will collaborate with training, inference, and product teams to deliver robust, observable systems and drive throughput for ML workloads.

Qualifications

  • Experience with Linux system administration, user and access management, and automation.
  • Scripting languages for system tooling (bash, Python, etc.).
  • Experience with containers (Docker) and scheduling/orchestration systems (Slurm, Kubernetes).
  • Experience running production services, including monitoring and alerting (Prometheus, Grafana), deployment pipelines, and incident response.
  • Experience building tooling for cluster validation and reliability (accelerator, networking, and storage health checks).
  • Experience setting up and managing developer tools and third-party services (cloud storage providers, Docker Hub, Slack, Gmail, Telegraf, experiment trackers).
  • Excellent debugging and troubleshooting skills across compute, storage, and networking.
  • Strong communication skills and ability to collaborate across technical and non-technical teams.

Responsibilities

  • Maintain and build core infrastructure for model training and inference cloud.
  • Administer Linux clusters, user onboarding/offboarding, access control, and security auditing.
  • Monitor resources, job scheduling, and cluster health; validate new hardware.
  • Support developer workflows (VSCode remote development, Docker).
  • Own large-scale training job enabling and scaling for inference platform.
  • Oversee service reliability: uptime, incident response, and postmortems.

Skills

Linux system administration
Automation scripting (bash, Python)
Docker
Kubernetes
Slurm

Tools

Prometheus
Grafana
VSCode Remote Development

Job description

Zyphra Technologies Inc. seeks a Machine Learning Systems Administrator/DevOps Engineer to maintain core infrastructure for model training and inference.

The role spans large training clusters and production inference services across heterogeneous accelerators, focusing on reliability, security, and scalable workflows. You will collaborate with training, inference, and product teams to deliver robust, observable systems and drive throughput for ML workloads.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

ML Systems Engineer: Training Clusters & Inference Cloud
ML Systems Engineer: Training Clusters & Inference Cloud

Bison • San Francisco (CA)

On-site
USD 180,000 - 260,000
Comprehensive medical, dental, vision
Competitive compensation
Relocation and immigration support
+3
ML Infra & DevOps Engineer: Training & Inference Cloud
ML Infra & DevOps Engineer: Training & Inference Cloud

Zyphra • San Francisco (CA)

On-site
USD 190,000 - 260,000
Medical, dental, vision plans
401(k) plan
Relocation support
+3
Machine Learning Systems Administrator / DevOps Engineer: Training Clusters & Inference Cloud
Machine Learning Systems Administrator / DevOps Engineer: Training Clusters & Inference Cloud

Zyphra Technologies Inc. • San Francisco (CA)

On-site
USD 180,000 - 230,000
Comprehensive medical, dental, vision
401(k) plan
Relocation and immigration support
+3
Machine Learning Systems Administrator / DevOps Engineer: Training Clusters & Inference Cloud
Machine Learning Systems Administrator / DevOps Engineer: Training Clusters & Inference Cloud

Bison • San Francisco (CA)

On-site
USD 180,000 - 260,000
Comprehensive medical, dental, vision
Competitive compensation
Relocation and immigration support
+3
Platform Engineer: ML Infra, Reliability & Scale
Platform Engineer: ML Infra, Reliability & Scale

Zyphra Technologies Inc. • San Francisco (CA)

On-site
USD 150,000 - 210,000
Comprehensive medical, dental, vision
401(k) plan
Relocation and immigration support
+3
ML Systems Engineer: Training & Inference Infra
ML Systems Engineer: Training & Inference Infra

NextToken AI Inc. • Mountain View (CA)

On-site
USD 180,000 - 250,000
ML Systems Engineer: Scalable Training & Inference
ML Systems Engineer: Scalable Training & Inference

IMC • New York (NY)

On-site
USD 175,000 - 250,000
ML Systems Engineer — Training & Inference Infrastructure
ML Systems Engineer — Training & Inference Infrastructure

NextToken • Mountain View (CA)

On-site
USD 190,000 - 240,000
ML Systems Engineer: Scalable Training & Inference
ML Systems Engineer: Scalable Training & Inference

IMC Trading • New York (NY)

On-site
USD 175,000 - 250,000
Platform Engineer - AI Infra, ML Deployment, Unlimited PTO
Platform Engineer - AI Infra, ML Deployment, Unlimited PTO

Bison • San Francisco (CA)

On-site
USD 140,000 - 230,000
Medical, dental, vision
401(k) plan
Relocation and immigration support
+3