AI Research Engineer | Remote

Crossing Hurdles

United States

Remote

USD 300,000 - 700,000

Full time

5 days ago
Be an early applicant

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Job summary

Crossing Hurdles is seeking a Member of Technical Staff for the Cortex Lab to design evaluation frameworks and benchmarks for LLM-powered agents. This remote role expects 10-40 hours per week and targets high-impact AI system reliability across production workflows.

You will develop datasets, run rapid experiments, and collaborate with researchers and enterprise customers to optimize AI systems and workflows. Strong Python and ML background required.

Qualifications

  • Strong background in applied AI research or machine learning.
  • Experience designing evaluations for AI agents and multi-step workflows.
  • Strong Python skills and experience building evaluation pipelines.
  • Experience developing benchmarks and evaluation metrics for AI systems.

Responsibilities

  • Design and implement evaluation frameworks, benchmarks, and datasets for LLM-powered agents and complex AI workflows.
  • Identify, analyze, and prioritize model, agent, and workflow failures in real-world production environments.
  • Run rapid experiments to validate hypotheses, measure performance, and improve agent reliability and effectiveness.
  • Partner with enterprise customers and researchers to evaluate, deploy, and optimize AI systems across production workflows.
  • Develop high-quality training and evaluation data, task specifications, and feedback mechanisms to improve system performance.
  • Build Python-based evaluation pipelines, experimentation tools, and lightweight research infrastructure.

Skills

Applied AI research
Evaluation pipelines
LLM systems
Python programming

Tools

Python

Job description

Position: Member of Technical Staff, Cortex Lab

Type: Full-time

Compensation: $300K - $700K/yr

Location: Remote

Commitment: 10-40 hrs/week

Role Responsibilities
  • Design and implement evaluation frameworks, benchmarks, and datasets for LLM-powered agents and complex AI workflows.
  • Identify, analyze, and prioritize model, agent, and workflow failures in real-world production environments.
  • Run rapid experiments to validate hypotheses, measure performance, and improve agent reliability and effectiveness.
  • Partner with enterprise customers and researchers to evaluate, deploy, and optimize AI systems across production workflows.
  • Develop high-quality training and evaluation data, task specifications, and feedback mechanisms to improve system performance.
  • Build Python-based evaluation pipelines, experimentation tools, and lightweight research infrastructure.
Requirements
  • Strong background in applied AI research, research engineering, machine learning, or LLM-based systems.
  • Hands-on experience designing and implementing evaluations for AI agents, including tool use, multi-step workflows, and task completion.
  • Strong Python programming skills and experience building evaluation pipelines, experimental tooling, or ML infrastructure.
  • Experience developing benchmarks, datasets, evaluation metrics, and experimental methodologies for AI systems.
Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Remote AI Research Engineer: Evaluation & ML Infra
Remote AI Research Engineer: Evaluation & ML Infra

Crossing Hurdles • United States

Remote
USD 300,000 - 700,000
Member of Technical Staff, Cortex Lab (applied AI research)
Member of Technical Staff, Cortex Lab (applied AI research)

Confidential • United States

Remote
USD 200,000 - 250,000
Equity compensation
Performance-based bonuses
Health insurance reimbursement
+2
Remote AI Research Engineer — Agent & LLM Systems
Remote AI Research Engineer — Agent & LLM Systems

Softechub • United States

Remote
USD 138,000 - 179,000
Remote Applied AI Staff Engineer — Agent Evaluation
Remote Applied AI Staff Engineer — Agent Evaluation

Confidential • United States

Remote
USD 200,000 - 250,000
Equity compensation
Performance-based bonuses
Health insurance reimbursement
+2
Senior AI Researcher, Evaluation & Agent Performance
Senior AI Researcher, Evaluation & Agent Performance

Cobalt • New York (NY)

On-site
USD 160,000 - 230,000
Member of Technical Staff, Cortex Lab (applied AI research)
Member of Technical Staff, Cortex Lab (applied AI research)

Softechub • United States

Remote
USD 138,000 - 179,000
AI Research Scientist (United States, Remote)
AI Research Scientist (United States, Remote)

Rex.zone • United States

Remote
USD 41,328 - 68,880
Member of Technical Staff, Cortex Lab (applied AI research)
Member of Technical Staff, Cortex Lab (applied AI research)

Frontier AI Lab • United States

Remote
USD 200,000 - 250,000
Equity compensation
Health insurance reimbursement
401(K) match
+1
Member of Technical Staff - AI Research
Member of Technical Staff - AI Research

Acceler8 Talent • Massachusetts

Hybrid
USD 140,000 - 240,000
AI Analyst | Remote
AI Analyst | Remote

Crossing Hurdles • United States

Remote
USD 240,000 - 350,000