Remote ML Engineer - LLM Evaluation & Agentic Benchmarks

Turing

United States

Remote

USD 34,000 - 55,000

Full time

14 days+

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Benefits offered by this job

Fully remote work

Job summary

Turing is hiring ML Engineers to join its AI Evaluation team as contractors. You will design and run evaluation suites for LLMs, benchmark agentic workflows, and analyze results to improve frontier AI systems.

This fully remote role requires strong Python skills, experience with LLM APIs, and the ability to work across time zones with PST overlap. Engagement is a contractor assignment for a duration of 3 months, with a minimum of 20 hours per week and overlap of at least 5 hours per day with

Qualifications

  • Currently pursuing or recently completed a Bachelor's, Master’s, or PhD in Computer Science, Machine Learning, Data Science, Mathematics, or a related quantitative field.
  • Strong programming skills in Python, including NumPy, Pandas, Jupyter.
  • Working understanding of how modern LLMs are trained, prompted, and evaluated.
  • Experience interacting with LLM APIs (OpenAI, Anthropic, Google, Hugging Face, vLLM).
  • Strong analytical and writing skills—ability to document tasks, results, and recommendations.
  • Comfortable with fully remote work and overlapping collaboration hours.

Responsibilities

  • Design, build, and run evaluation suites for LLMs across reasoning, coding, math, tool use, and long-context tasks.
  • Develop benchmarks and rubrics for agentic workflows, including multi-step planning and tool calling.
  • Curate, clean, and annotate evaluation datasets; write clear task specs and grading criteria.
  • Run model comparisons and analyze results to produce structured reports with actionable insights.
  • Implement automated graders and validate them against human judgments.

Skills

Python
NumPy
Pandas
Jupyter
LLM APIs
Analytical writing
Remote work

Education

Bachelor's/Master's/PhD in CS/ML/Data Science

Tools

LangGraph
AutoGen
CrewAI
OpenAI Agents SDK

Job description

Turing is hiring ML Engineers to join its AI Evaluation team as contractors. You will design and run evaluation suites for LLMs, benchmark agentic workflows, and analyze results to improve frontier AI systems.

This fully remote role requires strong Python skills, experience with LLM APIs, and the ability to work across time zones with PST overlap. Engagement is a contractor assignment for a duration of 3 months, with a minimum of 20 hours per week and overlap of at least 5 hours per day with

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

ML Engineer - LLM & Agentic Evaluations
ML Engineer - LLM & Agentic Evaluations

Turing • United States

Remote
USD 34,000 - 55,000
Fully remote work
Remote ML Engineer - LLM & Agentic Evaluations (Contractor)
Remote ML Engineer - LLM & Agentic Evaluations (Contractor)

AuraOne Human Data • United States

Remote
USD 83,000 - 165,000
Senior Software Engineer - Remote LLM Eval
Senior Software Engineer - Remote LLM Eval

Turing • United States

Remote
USD 41,000 - 83,000
Fully remote
Remote AI & ML Engineer for LLM Training & Evaluation
Remote AI & ML Engineer for LLM Training & Evaluation

Prolific • United States

Remote
USD 91,000 - 129,000
Remote AI/ML Engineer: LLM Agents & Backend
Remote AI/ML Engineer: LLM Agents & Backend

24-MAG • United States

Remote
USD 220,000 - 300,000
Fully remote
Remote Senior Software Engineer, LLM Evaluation & QA Tools
Remote Senior Software Engineer, LLM Evaluation & QA Tools

OpenTrain AI, Inc. • United States

Remote
USD 83,000 - 124,000
Remote AI/ML Engineer — LLMs, RAG & MLOps
Remote AI/ML Engineer — LLMs, RAG & MLOps

Gain America Inc. • United States

Remote
USD 110,000 - 207,000
Remote AI Training Engineer (LLM Evaluation & RLHF)
Remote AI Training Engineer (LLM Evaluation & RLHF)

Prolific • United States

Remote
USD 91,000 - 129,000
ML Evaluation Scientist - LLM Benchmarks
ML Evaluation Scientist - LLM Benchmarks

Scale AI, Inc. • Seattle (WA)

On-site
USD 181,000 - 226,000
Health coverage
Retirement benefits
L&D stipend
+2
Remote AI/ML Engineer — LLM Training & Evaluation
Remote AI/ML Engineer — LLM Training & Evaluation

Rex.zone • United States

Remote
USD 41,328 - 68,880