Engineering Manager, Kubernetes Bare Metal Infrastructure

FyndMyAI

San Francisco (CA)

On-site

USD 300,000 - 350,000

Full time

12 days ago

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Benefits offered by this job

Health insurance
Startup equity
Benefits

Job summary

Together AI is hiring an Engineering Manager/Tech Lead Manager to run the Kubernetes on bare-metal GPU infrastructure across our data centers. You will own provisioning, upgrades and health checks for thousands of GPUs, ensuring reliability for training, fine-tuning and serving models.

You will set technical direction, drive automation, and lead a team of engineers, coordinating with data center operations, hardware vendors and product teams.

Qualifications

  • 8+ years in infrastructure, SRE or platform engineering
  • 2+ years managing engineers
  • Deep hands-on experience running Kubernetes in production at scale
  • Grounding in Linux systems, networking and hardware (BIOS/BMC, PXE)

Responsibilities

  • Own the bare-metal Kubernetes platform across data centers and regions
  • Drive automated provisioning of GPU nodes from imaging to cluster join
  • Manage cluster lifecycle: upgrades, capacity, health checks, remediation
  • Prioritize GPU scheduling, topology-aware placement and high-performance networking
  • Define and meet SLOs for availability and utilization
  • Invest in observability across hardware, networking, Kubernetes and workloads
  • Build a healthy on-call culture with blameless postmortems

Skills

Kubernetes in prod
Infrastructure leadership
Linux systems
Networking
Incident response
On-call/SLOs
Technical judgment
Communication

Tools

Cluster API
Metal3
Ironic
MAAS
Ansible
Terraform

Job description

# Engineering Manager, Kubernetes Bare Metal InfrastructureTogether AIEngineering · Manager · Full-time · San Francisco, United StatesListed from the company's public job board. Apply on their site.ApplyTogether AIAbout The RoleWe are hiring an Engineering Manager/Tech Lead Manager to lead the team that runs Kubernetes on bare-metal GPU infrastructure at Together AI. You will own how thousands of GPUs across our data centers are provisioned, orchestrated and kept healthy for the customers training, fine-tuning and serving models on our platform.Together AI is the AI Native Cloud: a full-stack platform for running open and custom generative AI models, from GPU clusters to serverless inference. Our research and engineering teams contribute widely used open-source work such as FlashAttention, and our infrastructure runs at the frontier of GPU hardware.This team sits at the base of that stack. Every training run, inference endpoint and customer cluster depends on the reliability, utilization and speed of the bare-metal Kubernetes layer you will lead.ResponsibilitiesOwn the bare-metal Kubernetes platformSet the technical direction for how we run Kubernetes on bare metal across multiple data centers and regions.Drive automated, zero-touch provisioning of GPU nodes, from firmware and OS imaging through cluster join and validation.Own cluster lifecycle at scale: upgrades, capacity expansion, node health checks, automated remediation and decommissioning.Make GPU scheduling, topology-aware placement and high-performance networking (InfiniBand, RoCE, RDMA) first-class in the platform.Drive reliability and efficiencyDefine and meet SLOs for cluster availability, node bring-up time and GPU utilization.Invest in observability across hardware, networking, Kubernetes and workloads so failures are found before customers see them.Cut the time from racked hardware to customer-ready capacity.Lead and grow the teamHire, manage and develop a team of infrastructure and platform engineers, setting a high bar for technical depth and operational ownership.Run the team's planning, prioritization and delivery, balancing roadmap work against fleet growth and production support.Build a healthy on-call culture: sustainable rotations, blameless postmortems and follow-through on remediation.Work with data center operations, networking, hardware vendors, inference, training and product teams to plan capacity and launch new hardware generations.Represent the platform's needs and trade-offs to leadership, and turn customer requirements into a clear roadmap.Requirements8+ years in infrastructure, SRE or platform engineering, including 2+ years managing engineers.Deep hands-on experience running Kubernetes in production at scale, ideally on bare metal rather than only managed services.Solid grounding in Linux systems, networking and hardware: kernels, drivers, BIOS/BMC, PXE boot and out-of-band management.Experience building automation for fleet provisioning and lifecycle with tools such as Cluster API, Metal3, Ironic, MAAS, Ansible or Terraform.A track record of owning production systems with on-call, SLOs and incident response.Strong technical judgment: you can review designs, debug alongside the team and make clear trade-offs.Clear communication with engineers, leadership and external partners.Nice to haveExperience operating large NVIDIA GPU clusters (H100, H200, B200, GB200) and the NVIDIA GPU and Network Operators, DCGM and NCCL.Familiarity with InfiniBand, RoCE, SR-IOV and high-performance storage for AI workloads.Experience with Slurm, Kueue, Volcano or other batch schedulers for training workloads.Contributions to Kubernetes or related CNCF projects.Experience at an AI lab, GPU cloud or hyperscaler.About Together AITogether AI, the AI Native Cloud, is purpose-built for AI engineers. AI application developers get high-performance inference that scales reliably, fine-tuning and reinforcement learning for creating frontier-level specialized models, and pre-training at massive scale for fully custom intelligence, all around a marketplace of leading open models that teams can run, adapt, and own. Trusted by Cursor, Decagon, ElevenLabs, Salesforce, and Zoom, Together serves 400+ trillion tokens a month.CompensationWe offer competitive compensation, startup equity, health insurance and other competitive benefits. The US base salary range for this full-time position is: $300,000 -350,000 + equity + benefits. Our salary ranges are determined by location, level and role. Individual compensation will be determined by experience, skills, and job-related knowledge.Application PolicyPlease read each role carefully and apply to the position best suited to you. Candidates are limited to 6 applications within a 90-day period, and applications beyond that limit will be automatically rejected. You may apply to additional roles 90 days after your last application.Equal OpportunityTogether AI is an Equal Opportunity Employer and is proud to offer equal employment opportunity to everyone regardless of race, color, ancestry, religion, sex, national origin, sexual orientation, age, citizenship, marital status, disability, gender identity, veteran status, and more.
Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Engineering Manager, Kubernetes Bare Metal Infrastructure
Engineering Manager, Kubernetes Bare Metal Infrastructure

Together AI • San Francisco (CA)

On-site
USD 300,000 - 350,000
Equity
Health insurance
Senior Software Engineer - AI Compute, Together Cloud
Senior Software Engineer - AI Compute, Together Cloud

FDE Times • San Francisco (CA)

Hybrid
USD 220,000 - 270,000
[Junior / Senior / Staff] Software Engineer, Inference / Compute Infrastructure Engineering
[Junior / Senior / Staff] Software Engineer, Inference / Compute Infrastructure Engineering

FDE Times • San Francisco (CA)

On-site
USD 160,000 - 280,000
Startup equity
Health insurance
Equity
Together AI Software Development 6 months ago AI Infrastructure Systems Engineer San Francisco, United States · Full-time $190k–$270k
Together AI Software Development 6 months ago AI Infrastructure Systems Engineer San Francisco, United States · Full-time $190k–$270k

Sptix AI LLP • San Francisco (CA)

On-site
USD 190,000 - 270,000
Health insurance
Startup equity
Competitive benefits
(Staff / Senior Staff) AI Infra Engineer, Inference Infrastructure
(Staff / Senior Staff) AI Infra Engineer, Inference Infrastructure

Together AI • San Francisco (CA)

On-site
USD 250,000 - 320,000
Startup equity
Health insurance
Competitive benefits
(Senior/Junior AI Inference Reliability Engineer, Compute Infra
(Senior/Junior AI Inference Reliability Engineer, Compute Infra

FDE Times • San Francisco (CA)

On-site
USD 160,000 - 280,000
Equity
Health insurance
AI Infrastructure System Engineer
AI Infrastructure System Engineer

Together AI • United States

Remote
USD 140,000 - 220,000
(Staff / Senior Staff) AI Inference Reliability Engineer, Compute Infra
(Staff / Senior Staff) AI Inference Reliability Engineer, Compute Infra

FDE Times • San Francisco (CA)

On-site
USD 250,000 - 320,000
Startup equity
Health insurance
Competitive benefits
Senior Backend Engineer, Inference Platform
Senior Backend Engineer, Inference Platform

FDE Times • San Francisco (CA)

On-site
USD 200,000 - 290,000
Startup equity
Health insurance
Staff Software Engineer - AI Compute, Together Cloud
Staff Software Engineer - AI Compute, Together Cloud

FDE Times • San Francisco (CA)

Remote
USD 260,000 - 300,000
Remote work flexibility