Infrastructure Engineer for Large-Scale ML Training

Thinking Machines Lab Inc.; San Francisco (CA); On-site; USD 300,000 - 400,000; Full time

Benefits offered by this job

Health, dental, and vision benefits

Unlimited PTO

Paid parental leave

Relocation support

Job summary

Thinking Machines is hiring a Software Engineer, Infrastructure to design and build the distributed systems powering our model training and serving platforms in San Francisco. You will own critical infrastructure used by research and product teams.

The role emphasizes large-scale systems, Python and Go, and collaboration with researchers and platform engineers. This is a fast-moving startup environment with real ownership and impact.

Qualifications

• Demonstrated expertise designing and developing large-scale distributed systems.
• Strong proficiency in Python and Go.
• Experience building, deploying, and operating production infrastructure at scale.
• Solid grounding in distributed systems fundamentals, such as consensus, consistency, fault tolerance, and networking.

Responsibilities

• Design, build, and operate distributed systems that support large-scale model training and inference across thousands of accelerators.
• Build and maintain core infrastructure, including orchestration, scheduling, storage, and resource management systems.
• Improve the reliability, performance, and observability of infrastructure used across research and product teams.
• Partner with researchers and platform engineers to understand infrastructure needs and turn them into robust, well-abstracted systems.
• Debug and resolve complex distributed failures across the stack, from networking and storage to compute and scheduling.
• Write and maintain internal libraries and APIs, primarily in Python and Go, that other engineers build on.

Skills

Distributed systems design

Python

Go

Production infra

Kubernetes

IaC

Tools

Kubernetes

Terraform

Job description

Thinking Machines is hiring a Software Engineer, Infrastructure to design and build the distributed systems powering our model training and serving platforms in San Francisco. You will own critical infrastructure used by research and product teams.

The role emphasizes large-scale systems, Python and Go, and collaboration with researchers and platform engineers. This is a fast-moving startup environment with real ownership and impact.