Senior GPU HPC Infra Engineer for AI Training

Showcify; United States; On-site; USD 180,000 - 240,000; Full time

Benefits offered by this job

Lunch stipend

Health & dental benefits

RRSP/401K

Parental leave

Enrichment benefits

6 weeks paid vacation

Office travel budget & home office

Job summary

Cohere is seeking a Staff Software Engineer to design and operate ML/HPC infrastructure at scale. You will deploy GPU/TPU superclusters, optimize cost and performance, and ensure reliable AI workloads across clouds.

You will collaborate with researchers to translate needs into robust infra, build self-service tools, and drive observability and IaC practices across the org. This requires strong systems programming and cross-team mentorship.

Qualifications

• Deep expertise in ML/HPC infrastructure and GPU/TPU clusters.
• Experience deploying distributed training with JAX, PyTorch, or TensorFlow.
• Ability to design scalable Kubernetes-based AI infra at scale.
• Proficiency in Python and Go for ML tooling and systems.
• Strong knowledge of Linux internals and RDMA networking.
• Track record working with AI researchers to solve infra challenges.
• Self-directed, proactive problem solving in fast-moving teams.

Responsibilities

• Build and scale ML-optimized HPC infrastructure and GPU/TPU clusters across clouds.
• Collaborate with cloud providers to optimize cost, reliability, and performance.
• Troubleshoot bottlenecks and minimize disruptions to AI/ML workflows.
• Create self-service tools and interfaces for researchers to monitor and debug jobs.
• Drive infrastructure innovation for distributed training and ML workloads.
• Promote observability, automation, and IaC across teams.
• Share knowledge via code reviews and documentation.

Skills

ML/HPC infra

Kubernetes at scale

Python

Go

Linux internals

RDMA networking

Research collaboration

Problem solving

Tools

JAX

PyTorch

TensorFlow

Job description

Cohere is seeking a Staff Software Engineer to design and operate ML/HPC infrastructure at scale. You will deploy GPU/TPU superclusters, optimize cost and performance, and ensure reliable AI workloads across clouds.

You will collaborate with researchers to translate needs into robust infra, build self-service tools, and drive observability and IaC practices across the org. This requires strong systems programming and cross-team mentorship.