Senior Systems Engineer - AI GPU Infra & Fault Tolerance

Clockwork.io; Palo Alto (CA); On-site; USD 150,000 - 210,000; Full time

Benefits offered by this job

Catered lunch

Equity program

Benefits package

Job summary

Clockwork.io in Palo Alto is building infrastructure for fault-tolerant, high-performance distributed GPU training. This systems role digs into internals, designing and implementing low-level software that runs at scale and improves reliability across runtimes and frameworks.

You will own the development from design to production, work with CUDA/NCCL and PyTorch internals, and help shape the next generation of distributed GPU systems at Clockwork.io.

Qualifications

• Must have systems-building experience (not just deployment).
• Experience with distributed storage, databases, or coordination systems.
• Familiarity with runtimes, profilers, or performance tooling.
• Experience with large-scale systems at the systems layer.

Responsibilities

• Design and implement low-level systems software for GPU clusters.
• Modify and extend frameworks like PyTorch, NCCL, CUDA runtime.
• Build components to improve reliability and efficiency of large-scale GPU training.
• Debug complex distributed, concurrent systems where failures are subtle.
• Own systems end-to-end: from design through production.

Skills

C/C++

Concurrency

Distributed systems

Large codebases

Tools

CUDA

NCCL

PyTorch internals

RDMA

Job description

Clockwork.io in Palo Alto is building infrastructure for fault-tolerant, high-performance distributed GPU training. This systems role digs into internals, designing and implementing low-level software that runs at scale and improves reliability across runtimes and frameworks.

You will own the development from design to production, work with CUDA/NCCL and PyTorch internals, and help shape the next generation of distributed GPU systems at Clockwork.io.