Remote AI Research Engineer: Evaluation & ML Infra

Crossing Hurdles

United States

Remote

USD 300,000 - 700,000

Full time

5 days ago
Be an early applicant

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Job summary

Crossing Hurdles is seeking a Member of Technical Staff for the Cortex Lab to design evaluation frameworks and benchmarks for LLM-powered agents. This remote role expects 10-40 hours per week and targets high-impact AI system reliability across production workflows.

You will develop datasets, run rapid experiments, and collaborate with researchers and enterprise customers to optimize AI systems and workflows. Strong Python and ML background required.

Qualifications

  • Strong background in applied AI research or machine learning.
  • Experience designing evaluations for AI agents and multi-step workflows.
  • Strong Python skills and experience building evaluation pipelines.
  • Experience developing benchmarks and evaluation metrics for AI systems.

Responsibilities

  • Design and implement evaluation frameworks, benchmarks, and datasets for LLM-powered agents and complex AI workflows.
  • Identify, analyze, and prioritize model, agent, and workflow failures in real-world production environments.
  • Run rapid experiments to validate hypotheses, measure performance, and improve agent reliability and effectiveness.
  • Partner with enterprise customers and researchers to evaluate, deploy, and optimize AI systems across production workflows.
  • Develop high-quality training and evaluation data, task specifications, and feedback mechanisms to improve system performance.
  • Build Python-based evaluation pipelines, experimentation tools, and lightweight research infrastructure.

Skills

Applied AI research
Evaluation pipelines
LLM systems
Python programming

Tools

Python

Job description

Crossing Hurdles is seeking a Member of Technical Staff for the Cortex Lab to design evaluation frameworks and benchmarks for LLM-powered agents. This remote role expects 10-40 hours per week and targets high-impact AI system reliability across production workflows.

You will develop datasets, run rapid experiments, and collaborate with researchers and enterprise customers to optimize AI systems and workflows. Strong Python and ML background required.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

AI Research Engineer | Remote
AI Research Engineer | Remote

Crossing Hurdles • United States

Remote
USD 300,000 - 700,000
Remote Applied AI Staff Engineer — Agent Evaluation
Remote Applied AI Staff Engineer — Agent Evaluation

Confidential • United States

Remote
USD 200,000 - 250,000
Equity compensation
Performance-based bonuses
Health insurance reimbursement
+2
Remote AI Research Engineer — Agent & LLM Systems
Remote AI Research Engineer — Agent & LLM Systems

Softechub • United States

Remote
USD 138,000 - 179,000
Remote AI Evaluation Architect | Frontier ML Systems
Remote AI Evaluation Architect | Frontier ML Systems

Crossing Hurdles • United States

Remote
USD 240,000 - 350,000
Remote AI/ML Engineer: LLMs, APIs & Production Systems
Remote AI/ML Engineer: LLMs, APIs & Production Systems

Crossing Hurdles • United States

Remote
USD 220,000 - 300,000
Member of Technical Staff, Cortex Lab (applied AI research)
Member of Technical Staff, Cortex Lab (applied AI research)

Confidential • United States

Remote
USD 200,000 - 250,000
Equity compensation
Performance-based bonuses
Health insurance reimbursement
+2
Staff Research Engineer, Frontier AI & Evaluation (Remote)
Staff Research Engineer, Frontier AI & Evaluation (Remote)

Crossing Hurdles • United States

Remote
USD 48,000 - 103,000
Member of Technical Staff, Cortex Lab (applied AI research)
Member of Technical Staff, Cortex Lab (applied AI research)

Softechub • United States

Remote
USD 138,000 - 179,000
Frontier AI Research Engineer (Remote)
Frontier AI Research Engineer (Remote)

Crossing Hurdles • United States

Remote
USD 600,000 - 2,000,000
Remote AI/ML Engineer for Internal Platform Systems
Remote AI/ML Engineer for Internal Platform Systems

Crossing Hurdles • United States

Remote
USD 220,000 - 300,000
Remote work