Remote Senior Software Engineer: LLM Benchmarking & Infra

remote work; United States; Remote; USD 120,000 - 180,000; Full time

Job summary

Turing builds infrastructure for evaluating coding agents and language models. You will own features across the stack—from data models to the UI—shipping production-grade backend and frontend work for benchmark tasks and agent evaluations.

You will design end-to-end features, develop evaluation workflows, and ensure reliability as usage grows. Strong Python/Django and PostgreSQL are essential, with TS/React for user-visible changes.

Qualifications

• 5+ years building and operating production backend systems with ownership of at least one service.
• Bachelor's degree or higher in Software Engineering, Computer Science, or a related field.
• Strong Python and Django (or similar), PostgreSQL, and a task queue such as Celery.
• Hands-on experience in LLM evaluation and understanding benchmark task construction.
• Comfort with TypeScript and React frontend for user-visible changes.
• Nice to have experience with Docker, Kubernetes, or cloud sandboxes and LLM-as-a-judge tooling.

Responsibilities

• Design and ship end-to-end features including data models, APIs, background jobs, and UI elements when needed.
• Develop and improve agent and model evaluation workflows in sandboxed environments.
• Quality-check benchmark tasks using verifiers and rubric-based or LLM-based grading.
• Integrate new models, agents, and benchmarks with generalizable approaches.
• Maintain system performance and reliability as usage grows.
• Review teammates’ pull requests to uphold engineering guidelines.

Skills

Python

Django

PostgreSQL

Celery

LLM evaluation

Education

Bachelor's degree or higher in Software Engineering/CS

Tools

Docker

Kubernetes

Cloud sandboxes

LLM-as-a-judge tooling

Job description

Turing builds infrastructure for evaluating coding agents and language models. You will own features across the stack—from data models to the UI—shipping production-grade backend and frontend work for benchmark tasks and agent evaluations.

You will design end-to-end features, develop evaluation workflows, and ensure reliability as usage grows. Strong Python/Django and PostgreSQL are essential, with TS/React for user-visible changes.