Remote AI Evaluation Architect | Frontier ML Systems

Crossing Hurdles; United States; Remote; USD 240,000 - 350,000; Full time

Job summary

Crossing Hurdles is seeking a Member of Technical Staff for Frontier AI. This remote, full-time role centers on leading end-to-end research and evaluation, designing ML-oriented data systems, and validating signals that improve model performance.

You will work with researchers to calibrate quality early and push for decisive action in ambiguous environments. You will design datasets, evaluation schemas, and QA processes, translating real-world behavior into structured opportunities while

Qualifications

• Strong judgment around research signal quality and when work is externalized.
• Experience designing ML-oriented datasets, evaluation frameworks, and QA processes.
• Ability to translate messy, real-world system behavior into structured research opportunities.
• Comfort operating in ambiguity with ownership and decisive action.
• Excellent written and verbal communication, especially about tradeoffs and signal strength.

Responsibilities

• Own end-to-end research and evaluation initiatives: framing, data design, calibration, and signal validation.
• Design ML-oriented data systems with task definitions, annotation schemas, rubrics, incentives, and pipelines.
• Analyze model and system failures to identify root causes and opportunities for improvement.
• Translate ambiguous behavior into structured evaluation frameworks and data categories.
• Collaborate with researchers to calibrate quality early and raise signal strength.
• Iterate on evaluations, datasets, and feedback loops to improve system performance.

Skills

Research signal quality

ML data systems design

Evaluation frameworks

QA processes

Clear communication

Job description

Crossing Hurdles is seeking a Member of Technical Staff for Frontier AI. This remote, full-time role centers on leading end-to-end research and evaluation, designing ML-oriented data systems, and validating signals that improve model performance.

You will work with researchers to calibrate quality early and push for decisive action in ambiguous environments. You will design datasets, evaluation schemas, and QA processes, translating real-world behavior into structured opportunities while