Multimodal AI Evaluation Engineer - Safety & Calibration

Enari Inc. (dba Verdun); Los Angeles (CA); Hybride; USD 140 000 - 210 000; Plein temps

Résumé du poste

Enari Inc. (dba Verdun) seeks an engineer to own an evaluation harness for multimodal security data, ensuring the system’s evaluation quality matches model quality across diverse real-world footage.

You will design replayable pipelines, calibrate confidence, and build an active-learning loop from disagreements and operator overrides. The role emphasizes production-grade Python, data tooling, notebooks, and eval services, with a focus on rapid iteration for prompts and models in a

Qualifications

• Experience shipping applied ML or LLM systems where eval quality mattered as much as model quality.
• Strong experiment design, error analysis, and offline/online benchmarking skills.
• Strong statistical instincts around calibration, uncertainty, and asymmetric risk.

Responsabilités

• Ownership of a replayable evaluation harness for multimodal security incidents.
• Develop golden sets and sliced evals across environmental conditions and camera types.
• Maintain operator-parity benchmarks with recall on true threats as the primary metric.

Connaissances

Experiment design

Error analysis

Benchmarking

Python

Data tooling

Notebooks

Eval services

Multimodal models

Prompt iteration

Description du poste

Enari Inc. (dba Verdun) seeks an engineer to own an evaluation harness for multimodal security data, ensuring the system’s evaluation quality matches model quality across diverse real-world footage.

You will design replayable pipelines, calibrate confidence, and build an active-learning loop from disagreements and operator overrides. The role emphasizes production-grade Python, data tooling, notebooks, and eval services, with a focus on rapid iteration for prompts and models in a