AI Inference Infrastructure Engineer

Thinking Machines Lab Inc.; San Francisco, Northern (CA, KY); Hybrid; USD 350,000 - 475,000; Full time

Benefits offered by this job

Health, dental, vision benefits

Unlimited PTO

Parental leave

Relocation support

Job summary

Thinking Machines Lab Inc. in San Francisco, California is seeking an infrastructure research engineer to design, optimize, and scale the systems that power large AI models.

Your work will make inference faster, more cost-effective, more reliable, and more reproducible to enable our teams to focus on advancing model capabilities rather than managing bottlenecks. Our focus is on performant and efficient model inference both to power real-world applications and to accelerate research.

Qualifications

• Bachelor’s degree or equivalent in CS or engineering.
• Understanding of deep learning frameworks and system architectures.
• Experience with inference serving for throughput and latency.
• Collaborative across cross-functional teams.
• Bias for action and ability to ship across stacks where opportunity is spotted.
• Strong coding skills and debugging in large codebases.

Responsibilities

• Collaborate with researchers to bring models into production.
• Enable high-performance inference for novel architectures.
• Design tools and architectures improving performance and efficiency.
• Optimize codebase and compute fleets for GPU usage.
• Extend orchestration frameworks for distributed inference and large batches.
• Establish reliability and observability standards across the inference stack.
• Publish learnings via internal docs, open-source libraries, or technical reports.

Skills

Bachelor level CS/Engineering

Deep learning frameworks

Inference serving systems

Cross-functional collaboration

Bias for action

Strong coding and debugging

Education

Bachelor’s degree or equivalent

Tools

PyTorch

JAX

SGLang

vLLM

Kubernetes

Ray

SLURM

Job description

Thinking Machines Lab Inc. in San Francisco, California is seeking an infrastructure research engineer to design, optimize, and scale the systems that power large AI models.

Your work will make inference faster, more cost-effective, more reliable, and more reproducible to enable our teams to focus on advancing model capabilities rather than managing bottlenecks. Our focus is on performant and efficient model inference both to power real-world applications and to accelerate research.