Senior Software Engineer, AI Evaluation Platform

remote work; United States; Remote; USD 120,000 - 180,000; Full time

Job summary

Turing builds infrastructure for evaluating frontier AI systems. This role owns features across the stack that support data models, APIs, background workers, and UI for AI evaluation at scale.

Design end-to-end features, improve evaluation workflows, and generalize integration of new models and benchmarks. You will maintain performance and review PRs to uphold engineering standards as usage grows.

Qualifications

• 5+ years building and operating production backend systems with ownership of at least one service.
• Strong Python experience, plus Django or similar framework, PostgreSQL, and a task queue such as Celery.
• Hands on experience in LLM evaluation beyond simple API calls; understanding benchmark task construction and result aggregation.

Responsibilities

• Design and ship end to end features including data model, API, background jobs, and UI when needed.
• Improve evaluation workflows for agents and language models, including sandboxed trials and scoring results.
• Enhance benchmark task quality checks with verifiers, rubrics, and judge-based scoring.
• Generalize integration of new models, agents, and benchmarks without per item special cases.
• Maintain system performance and reliability as usage grows through efficient queries and safe migrations.
• Review teammates' pull requests and uphold engineering guidelines and codebase consistency.

Skills

Python

Backend ownership

LLM evaluation

TypeScript

React

Django

PostgreSQL

Celery

Tools

Celery

PostgreSQL

Job description

Turing builds infrastructure for evaluating frontier AI systems. This role owns features across the stack that support data models, APIs, background workers, and UI for AI evaluation at scale.

Design end-to-end features, improve evaluation workflows, and generalize integration of new models and benchmarks. You will maintain performance and review PRs to uphold engineering standards as usage grows.