Platform Engineer - AI Evaluation & Infra

Precision Labs; San Francisco, Northern (CA, KY); Hybrid; USD 300,000 - 475,000; Full time

Benefits offered by this job

Health benefits

Dental benefits

Vision benefits

Unlimited PTO

Parental leave

Relocation support

Job summary

Thinking Machines is seeking a researcher-engineer to design and build a scalable evaluation platform that enables authors to create, run, and analyze model evaluations end-to-end. You will work across Python frameworks, data pipelines, APIs, and user-facing apps to support researchers’ workflows.

You will collaborate with pre-training, post-training, and applied teams to improve how evaluations influence research decisions, through robust versioning, provenance, and reliable deployment.

Qualifications

• Bachelor's degree or equivalent practical experience in computer science, engineering, machine learning, or related field.
• Two years of post-grad work experience as a software engineer or ML engineer.
• Hands-on experience building or maintaining evaluations, benchmarks, graders, or model-quality systems for large language or multimodal models.
• Strong software engineering fundamentals and experience building reliable, maintainable systems.
• Proficiency in at least one backend programming language; Python and Rust preferred; React/TypeScript on frontend.
• Experience with databases, data pipelines, distributed systems, or other data-intensive infrastructure.
• Ability to own projects from inception to deployment and operation.
• Experience collaborating with cross-functional partners and subject-matter experts.

Responsibilities

• Design, build, and maintain the platform for authoring, running, tracking, and analyzing model evaluations used in day-to-day work and model releases.
• Work across evaluation libraries, distributed backend systems, data pipelines, APIs, and user-facing applications to deliver end-to-end capabilities.
• Build flexible abstractions for evaluation tasks, environments, graders, datasets, and model outputs without slowing rapid research.
• Ensure evaluation results are reproducible and trustworthy via versioning, provenance, observability, recovery, and quality controls.
• Partner with researchers to identify bottlenecks and turn bespoke workflows into self-serve systems across teams.
• Collaborate with Research Tooling and the engineering team behind Thinking Machines’ internal research platform.

Skills

Python

Rust

React

TypeScript

Distributed systems

Data pipelines

Back-end development

Model evaluation

Collaboration

Problem discovery

Education

Bachelor's degree or equivalent in CS/Engineering

Tools

PostgreSQL

APIs

Data visualization

Job description

Thinking Machines is seeking a researcher-engineer to design and build a scalable evaluation platform that enables authors to create, run, and analyze model evaluations end-to-end. You will work across Python frameworks, data pipelines, APIs, and user-facing apps to support researchers’ workflows.

You will collaborate with pre-training, post-training, and applied teams to improve how evaluations influence research decisions, through robust versioning, provenance, and reliable deployment.