Production Inference Engineer: Scale & Reliability

Bison; San Francisco (CA); On-site; USD 300,000 - 400,000; Full time

Benefits offered by this job

Health benefits

Dental benefits

Vision benefits

Unlimited PTO

Parental leave

Relocation support

Job summary

Thinking Machines is hiring a Software Engineer, Inference to own the reliability, scale, and efficiency of the systems that serve our AI models to real users. This role focuses on production-facing services, ensuring safe deployments, and maintaining performance across multi-tenant serving platforms.

You will bridge cutting-edge inference techniques with day-to-day operations, handling rollouts, capacity planning, incidents, and maintaining a fast-growing platform online in a San Francisco

Qualifications

• Experience operating large-scale, latency-sensitive production systems.
• Proficiency in Python and Go or another systems language.
• Experience with observability, monitoring, and incident response for production services.
• Strong understanding of distributed systems and how they fail at scale.

Responsibilities

• Operate and scale the production inference systems that serve live traffic, including Tinker's multi-tenant serving platform.
• Own the rollout process for new models, model versions, and inference optimizations, ensuring safe, incremental deployment to production.
• Build and improve observability, alerting, and capacity planning so the team can detect, diagnose, and resolve production issues quickly.
• Partner with inference and research teams to productionize new serving techniques without compromising reliability.
• Lead incident response for production inference issues, driving root cause analysis and durable fixes.
• Design for graceful degradation, failover, and redundancy so that serving stays resilient as usage grows.
• Manage capacity and cost tradeoffs for serving infrastructure as traffic and model sizes scale.

Skills

Production systems

Python

Go or systems language

Observability & incident response

Distributed systems understanding

Job description

Thinking Machines is hiring a Software Engineer, Inference to own the reliability, scale, and efficiency of the systems that serve our AI models to real users. This role focuses on production-facing services, ensuring safe deployments, and maintaining performance across multi-tenant serving platforms.

You will bridge cutting-edge inference techniques with day-to-day operations, handling rollouts, capacity planning, incidents, and maintaining a fast-growing platform online in a San Francisco