Senior Software Engineer – LLM Evaluation

Turing

United States

Remote

USD 55,000 - 96,000

Part time

14 days+

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Benefits offered by this job

Flexible engagement (10–40 hrs/week)
Contractor (no medical/paid leave)

Job summary

Turing, based in San Francisco, is seeking a Software Engineering evaluator to curate code examples and provide precise solutions across Python, JavaScript (ReactJS), C/C++, Java, Rust, and Go. You will collaborate with researchers to advance AI-driven coding solutions and evaluate AI-generated code for efficiency, scalability, and reliability.

You will build verification agents, hypothesize steps in the software engineering life cycle, and work with cross-functional teams to push

Qualifications

  • Several years of software engineering experience, including 2+ years at a top-tier product company (e.g., Google, Stripe, Amazon, Apple, Meta, Netflix, Microsoft, Datadog, Dropbox, Shopify, PayPal, IBM Research).
  • Strong expertise in building full-stack applications and deploying scalable, production-grade software using modern languages and tools.
  • Deep understanding of software architecture, design, development, debugging, and code quality/review assessment.
  • Excellent oral and written communication skills for clear, structured evaluation rationales.

Responsibilities

  • Curate code examples and build solutions across Python, JavaScript (including ReactJS), C/C++, Java, Rust, and Go.
  • Evaluate and refine AI-generated code for efficiency, scalability, and reliability.
  • Collaborate with cross-functional teams to enhance AI-driven coding solutions.
  • Build agents that verify code quality and identify error patterns.
  • Hypothesize steps in the software engineering cycle and evaluate model capabilities on them.
  • Design verification mechanisms to automatically verify a software engineering task.

Skills

Full-stack development
Software architecture
Code quality / review
Communication skills
Python
JavaScript
ReactJS
C/C++
Java
Rust
Go

Job description

About Turing

Based in San Francisco, California, Turing is the world’s leading research accelerator for frontier AI labs and a trusted partner for global enterprises deploying advanced AI systems. Turing supports customers in two ways: first, by accelerating frontier research with high-quality data, advanced training pipelines, plus top AI researchers who specialize in coding, reasoning, STEM, multilinguality, multimodality, and agents; and second, by applying that expertise to help enterprises transform AI from proof of concept into proprietary intelligence with systems that perform reliably, deliver measurable impact, and drive lasting results on the P&L

Role Overview

As a Software Engineering evaluator, you will create cutting-edge datasets for training, benchmarking, and advancing large language models, collaborating closely with researchers. This includes curating code examples, providing precise solutions, and making corrections in Python, JavaScript (including ReactJS), C/C++, Java, Rust, and Go; evaluating and refining AI-generated code for efficiency, scalability, and reliability; and working with cross-functional teams to enhance enterprise-level AI-driven coding solutions.

What does day-to-day life look like?
  • Working on AI model training initiatives by curating code examples, building solutions, and correcting code in Python, JavaScript (including ReactJS), C/C++, Java, Rust, and Go.
  • Evaluate and refine AI-generated code to ensure that it is efficient, scalable, and reliable.
  • Collaborate with cross-functional teams to enhance AI-driven coding solutions against industry performance benchmarks.
  • Build agents that can verify the quality of the code and identify error patterns.
  • Hypothetize on steps in the software engineering cycle (prototyping, architecture design, API design, production implementation, launch, experiments, monitoring, operation maintenance) and evaluate model capabilities on them
  • Design verification mechanisms that can automatically verify a solution to a software engineering task.
Requirements
  • Several years of software engineering experience, including 2+ years of continuous full-time experience at a top-tier product company (e.g., Google, Stripe, Amazon, Apple, Meta, Netflix, Microsoft, Datadog, Dropbox, Shopify, PayPal, IBM Research).
  • Strong expertise in building full-stack applications and deploying scalable, production-grade software using modern languages and tools.
  • Deep understanding of software architecture, design, development, debugging, and code quality/review assessment.
  • Excellent oral and written communication skills for clear, structured evaluation rationales.
Offer Details
  • Commitment: flexible engagement, minimum 10 hrs/week, up to 40 hrs/week (partial PST overlap required)
  • Type: Contractor (no medical/paid leave)
  • Duration:1 month (starting next week; potential extensions based on performance and fit)
Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Senior Software Engineer – LLM Evaluation
Senior Software Engineer – LLM Evaluation

Parailabs • San Francisco (CA)

On-site
USD 55,000 - 96,000
Senior Software Engineer – LLM Evaluation (US/Canada/WEU based)
Senior Software Engineer – LLM Evaluation (US/Canada/WEU based)

Aitrainergigs • San Francisco (CA), Town of Belgium (WI)

Hybrid
USD 55,000 - 117,000
Senior Software Engineer – LLM Evaluation (US-based)
Senior Software Engineer – LLM Evaluation (US-based)

Turing • United States

Remote
USD 55,000 - 110,000
Senior Software Engineer – LLM Evaluation
Senior Software Engineer – LLM Evaluation

HumanitApp • California

On-site
USD 83,000 - 124,000
Senior Python Engineer – LLM Evaluation (US-based)
Senior Python Engineer – LLM Evaluation (US-based)

Turing • United States

Remote
USD 69,000 - 96,000
Software Engineer – AI Research & Evaluation (US-based)
Software Engineer – AI Research & Evaluation (US-based)

Turing • United States

Remote
USD 83,000 - 131,000
Senior Python Engineer – LLM Evaluation (US-based)
Senior Python Engineer – LLM Evaluation (US-based)

Aitrainergigs • United States

On-site
USD 83,000 - 138,000
Senior Software Engineer – LLM Evaluation (US-based)
Senior Software Engineer – LLM Evaluation (US-based)

HumanitApp • San Francisco (CA)

On-site
USD 83,000 - 138,000
Remote Software Developer
Remote Software Developer

Turing • Chicago (IL)

Remote
USD 55,000 - 110,000
Flexible hours
Partial PST overlap
Remote Software Developer
Remote Software Developer

Turing • Pittsburgh

Remote
USD 55,000 - 96,000