AI Infrastructure Platform Engineer — GPU & Multi-Cloud

Mercor; San Francisco (CA); Vor Ort; USD 180.000 - 280.000; Vollzeit

Benefits dieser Stelle

Bi-annual performance bonus structure

Generous equity grant vested over 4 yr

Up to $15k Relocation bonus

$10K housing bonus (0.5 miles of our o

$1.5K monthly stipend for meals

Free Equinox membership

$200 monthly laundry reimbursement

$200 monthly personal wellness reimb

Health, Dental, Vision insurance

Zusammenfassung

Mercor is hiring a software engineer to own the accelerator platform end-to-end. You will operate GPU clusters across AWS, GCP, neoclouds and on-prem data centers, with custom Kubernetes and Slurm operators.

The role focuses on design, deployment, and on-call ownership to keep accelerators busy and costs under control. You will build a high-throughput vLLM serving stack, implement auto-scaling, and create tooling for benchmarking, observability, and cost attribution.

Qualifikationen

• 5+ years building and operating production distributed systems or ML infrastructure.
• Proficient in Python and at least one of Go or C++.
• Deep Kubernetes experience and HPC tools (Slurm).
• Experience running GPU workloads at scale and multi-cloud infra.

Aufgaben

• Design, deploy and operate GPU clusters across AWS, GCP, hyperscalers, neoclouds and on-prem data centers, including custom Kubernetes and Slurm operators.
• Build scheduling to maximize accelerator utilization with gang scheduling, bin-packing and quotas.
• Run distributed training infrastructure using PyTorch Distributed, Ray, or Megatron.
• Deploy and tune high-throughput LLM serving with vLLM, including quantization and cache strategies.
• Create benchmarking and traffic-replay tools to measure latency and throughput and set baselines.
• Implement health checks and automated remediation for fault storms without human intervention.
• Instrument fleet utilization with DCGM, Prometheus and Grafana; write monitoring rules.
• Ingest utilization and billing data, support capacity planning and autoscaling initiatives.
• Prepare runbooks and internal docs; communicate tradeoffs to researchers, finance and leadership.

Kenntnisse

Distributed systems

Production systems

Python

Go

C++

Kubernetes

Git/shell

GPU workloads

Tools

Kubernetes

Slurm

Prometheus

Grafana

DataDog

CUDA

Triton

Jobbeschreibung

Mercor is hiring a software engineer to own the accelerator platform end-to-end. You will operate GPU clusters across AWS, GCP, neoclouds and on-prem data centers, with custom Kubernetes and Slurm operators.

The role focuses on design, deployment, and on-call ownership to keep accelerators busy and costs under control. You will build a high-throughput vLLM serving stack, implement auto-scaling, and create tooling for benchmarking, observability, and cost attribution.