Senior Software Engineer, AI Evaluation Infrastructure (Fully Remote)

Turing; United States; Remote; USD 150,000 - 210,000; Full time

Job summary

Turing in San Francisco seeks a backend engineer to design and ship features end to end, including data models, APIs, background workers, and UI when needed.

You will evaluate agents and models, run sandboxed trials, score results, and turn thousands of runs into trusted outcomes, contributing to AI benchmark quality. The role emphasizes ownership and pragmatic tradeoffs in an AI-forward culture.

Qualifications

• 5+ years building and operating production backend systems, with ownership of at least one service from design through on-call.
• Strong Python, experience with Django or a similar framework, PostgreSQL, and a task queue such as Celery.
• Hands-on experience in LLM evaluation, good tradeoff judgment, ownership, and comfort with TypeScript and React when needed.

Responsibilities

• Design and ship features end to end: data model, API, background workers, and the UI when needed.
• Build and improve how agents and models are evaluated: running trials in sandboxed environments, scoring results, and turning thousands of runs into trusted outcomes.
• Improve benchmark task quality checks, including verifiers, rubric-based and LLM-as-a-judge grading, and task fairness.

Skills

Python

Django

PostgreSQL

Celery

TypeScript

React

LLM evaluation

ownership

Tools

Celery

Job description

• Design and ship features end to end: data model, API, background workers, and the UI when needed.
• Build and improve how agents and models are evaluated: running trials in sandboxed environments, scoring results, and turning thousands of runs into trusted outcomes.
• Improve benchmark task quality checks, including verifiers, rubric-based and LLM-as-a-judge grading, and task fairness.

What you bring:

• Typically 5+ years building and operating production backend systems, with ownership of at least one service from design through on-call.
• Strong Python, experience with Django or a similar framework, PostgreSQL, and a task queue such as Celery.
• Hands-on experience in LLM evaluation, good tradeoff judgment, ownership, and comfort with TypeScript and React when needed.

How we work:

• Written engineering guidelines applied consistently, with the same review bar for everyone.
• Business logic lives in services, hot paths stay cheap, and lint/tests pass before a PR is ready.
• We prefer deleting code to adding it, and simple designs to clever ones.

Values:

• Client first: putting clients at the center of everything we do.
• Start-up speed: moving fast, staying agile, and favoring action.
• AI forward: helping clients build the future of AI and amplifying productivity with AI in our own workflows.

Turing

Turing accelerates superintelligence by providing high-quality datasets, reinforcement learning environments, and research benchmarks to frontier AI labs, and also deploys agentic AI systems for Fortune 500 enterprises. Headquartered in San Francisco, it operates with startup speed and an AI-forward culture.