AI Evaluation Engineer — Experiments & Benchmarks

Dyssonance AI; San Francisco (CA); Sur place; USD 180 000 - 240 000; Plein temps

Résumé du poste

Dyssonance AI seeks an Evals Engineer to shape how we measure progress toward a fundamentally different AI. You will work at the intersection of research, engineering, and experimental science to translate novel capabilities into falsifiable hypotheses and to design benchmarks when existing ones fall short.

You will establish meaningful baselines, build evaluation infrastructure, and influence research directions by exposing confounders and designing robust experiments that reveal why systems

Qualifications

• Strong software engineering skills, particularly in Python and modern ML tooling.
• Experience designing evaluations for machine learning systems, language models, or related tech.
• Excellent experimental-design instincts, including controls, ablations, confounders, and reproducibility.
• Experience developing datasets, benchmarks, evaluation protocols, or annotation methodologies.
• Strong quantitative reasoning and the ability to distinguish meaningful evidence from misleading metrics.
• Ability to translate ambiguous research questions into concrete experiments and working software.
• Clear technical communication skills to explain experiments and next steps.

Responsabilités

• Invent new ways to measure intelligence through evaluations for stateful reasoning, coherence, and continual reasoning under uncertainty.
• Turn research claims into testable hypotheses with observable behaviors and measurable criteria.
• Build novel benchmarks and datasets to evaluate emerging capabilities.
• Establish rigorous baselines by comparing against frontier models and ablations.
• Design experiments that explain results and identify underlying mechanisms.

Connaissances

Experiment design

Data analysis

Clear technical communication

Research-to-software translation

Outils

Python

ML tooling

Description du poste

Dyssonance AI seeks an Evals Engineer to shape how we measure progress toward a fundamentally different AI. You will work at the intersection of research, engineering, and experimental science to translate novel capabilities into falsifiable hypotheses and to design benchmarks when existing ones fall short.

You will establish meaningful baselines, build evaluation infrastructure, and influence research directions by exposing confounders and designing robust experiments that reveal why systems