Remote Senior AI Evaluation Infrastructure Engineer

turing; San Francisco (CA); Remote; USD 170,000 - 200,000; Full time

Benefits offered by this job

Equity

Job summary

Turing, headquartered in San Francisco, seeks a Senior Software Engineer to own AI evaluation infrastructure. You will design data models, APIs, background jobs, and UI elements, ensuring scalable evaluation of coding agents and LLMs.

The role emphasizes independent work, sound tradeoffs, and safe deployments. The position values hands-on experience with Django, Python, PostgreSQL, and Celery, plus familiarity with frontend tech.

Qualifications

• 5+ years building and operating production backend systems.
• Bachelor’s degree or higher in Software Engineering, Computer Science, or a related field.
• Strong Python and experience with Django, PostgreSQL, and Celery.
• Hands-on experience in LLM evaluation and benchmark tasks.
• Ability to spot ambiguous instructions and design robust solutions.
• Ownership mindset with end-to-end delivery including tests and migrations.

Responsibilities

• Design and ship features end to end: data model, API, workers, UI.
• Improve evaluation of agents and models with scalable runs and scoring.
• Quality-check benchmark tasks using verifiers and rubric-based grading.
• Integrate new models/agents/benchmarks with generalizable design.
• Maintain speed and reliability as usage grows with efficient queries and migrations.
• Review teammates’ PRs and uphold engineering guidelines.

Skills

Backend systems

Python

Django

PostgreSQL

LLM evaluation

TypeScript/React (frontend familiarity

Education

Bachelor’s degree in Software Engineering or Computer Science

Tools

Celery

Harbor

Terminal-Bench

SWE-bench

Inspect

Job description

Turing, headquartered in San Francisco, seeks a Senior Software Engineer to own AI evaluation infrastructure. You will design data models, APIs, background jobs, and UI elements, ensuring scalable evaluation of coding agents and LLMs.

The role emphasizes independent work, sound tradeoffs, and safe deployments. The position values hands-on experience with Django, Python, PostgreSQL, and Celery, plus familiarity with frontend tech.