Senior Applied Scientist

Ll Oefentherapie; Denver (CO); On-site; USD 150,000 - 210,000; Full time

Job summary

Ll Oefentherapie in Denver is seeking a Senior Applied Scientist to independently own complex evaluation work from problem definition through final recommendation. You will translate ambiguous product questions into measurable hypotheses, select or create benchmarks, design experiments, build evaluation pipelines, validate data and metrics, analyze failure modes, and communicate conclusions to science, engineering, product, and leadership stakeholders.

This is hands-on applied science.

Responsibilities

• Own complex evaluation work from problem definition through final recommendation.
• Translate ambiguous product questions into measurable hypotheses and design experiments.
• Collaborate with scientists, software engineers, product teams, data and human-annotation teams, and external partners to deliver evaluation results.

Job description

The OCI AI Evaluation team builds the evidence behind model-selection, product-readiness, and launch decisions. We evaluate frontier foundation models and AI systems across capabilities such as reasoning, coding and agentic coding, retrieval-augmented generation, AI agents, NL2SQL, multimodal understanding, multilingual performance, and responsible AI.

As a Senior Applied Scientist on the team, you will independently own complex evaluation work from problem definition through final recommendation. You will translate ambiguous product and customer questions into measurable hypotheses, select or create appropriate benchmarks, design experiments, build evaluation pipelines, validate data and metrics, analyze failure modes, and communicate conclusions to science, engineering, product, and leadership stakeholders.

This is hands-on applied science. You will write high-quality code, work with large and imperfect datasets, develop and calibrate automated evaluators, and turn one-off analyses into reproducible evaluation protocols and reusable infrastructure. You will examine more than aggregate benchmark scores, considering factors such as statistical validity, data provenance, contamination, robustness, cost, latency, reliability, safety, and operational constraints.
The work sits at the point where research results become product decisions. Success requires scientific rigor, strong engineering judgment, clear writing, and the ability to make progress when requirements, model access, data, or infrastructure are still evolving. You will collaborate closely with other scientists, software engineers, product teams, data and human-annotation teams, and external partners to deliver evaluation results that are technically defensible and useful in practice.