Senior AI Evaluation Infrastructure Engineer (Remote)

Turing; United States; Remote; USD 180,000 - 260,000; Full time

Job summary

Turing is seeking a Senior Software Engineer to build and maintain AI evaluation infrastructure. You will own end-to-end features from data model to UI, enabling scalable evaluation of coding agents and language models.

You’ll work independently, turning loosely defined requests into shipped, tested, and reliable solutions. You will collaborate across data, API, and frontend layers, improving verification, scoring, and benchmarking workflows while maintaining performance as user demand grows.

Qualifications

• 5+ years building and operating production backend systems.
• Strong Python and backend stack experience.
• Hands-on experience in LLM evaluation beyond API calls.
• Experience with scalable evaluation and benchmarking.
• Ability to design end-to-end features across data model, API, workers, and UI.
• Proven on-call ownership and system migrations/tests.

Responsibilities

• Design and ship features end to end: data model, API, background workers, and UI when needed.
• Improve evaluation of agents and models and turn trials into trusted results.
• Quality-check benchmark tasks using verifiers and rubric-based grading.
• Generalize integrations for new models, agents, and benchmarks.
• Maintain speed and reliability with scalable queries and migrations.
• Review PRs following engineering guidelines and maintain code quality.

Skills

Python

Django

PostgreSQL

Celery

LLM evaluation

Quality judgment

Ownership

TypeScript/React

Writing

Team conventions

Tools

Docker

Kubernetes

Modal

Daytona

Harbor

Inspect

SWE-bench

Terminal-Bench

Job description

Turing is seeking a Senior Software Engineer to build and maintain AI evaluation infrastructure. You will own end-to-end features from data model to UI, enabling scalable evaluation of coding agents and language models.

You’ll work independently, turning loosely defined requests into shipped, tested, and reliable solutions. You will collaborate across data, API, and frontend layers, improving verification, scoring, and benchmarking workflows while maintaining performance as user demand grows.