Production ML Inference Engineer — Unlimited PTO

Thinking Machines Lab Inc.; San Francisco (CA); On-site; USD 300,000 - 400,000; Full time

Benefits offered by this job

Health benefits

Dental & vision

Unlimited PTO

Parental leave

Relocation support

Job summary

Thinking Machines Lab Inc. is hiring a Software Engineer, Inference to own the reliability, scale, and efficiency of serving our AI models to real users in a production environment.

You will bridge cutting-edge inference techniques with real traffic, rolling out new models safely and improving observability. The role focuses on multi-tenant serving, capacity planning, and incident response to keep the platform fast and resilient as usage grows.

Qualifications

• Experience operating large-scale, latency-sensitive production systems.
• Proficiency in Python and Go or another systems language.
• Experience with observability, monitoring, and incident response for production services.
• Strong understanding of distributed systems and how they fail at scale.

Responsibilities

• Operate and scale the production inference systems that serve live traffic, including Tinker's multi-tenant serving platform.
• Own the rollout process for new models, model versions, and inference optimizations, ensuring safe, incremental deployment to production.
• Build and improve observability, alerting, and capacity planning so the team can detect, diagnose, and resolve production issues quickly.
• Partner with inference and research teams to productionize new serving techniques without compromising reliability.
• Lead incident response for production inference issues, driving root cause analysis and durable fixes.
• Design for graceful degradation, failover, and redundancy so that serving stays resilient as usage grows.
• Manage capacity and cost tradeoffs for serving infrastructure as traffic and model sizes scale.

Skills

Python

Go or systems language

Observability

Distributed systems

Job description

Thinking Machines Lab Inc. is hiring a Software Engineer, Inference to own the reliability, scale, and efficiency of serving our AI models to real users in a production environment.

You will bridge cutting-edge inference techniques with real traffic, rolling out new models safely and improving observability. The role focuses on multi-tenant serving, capacity planning, and incident response to keep the platform fast and resilient as usage grows.