Applied AI / Evals Engineer

Enari Inc. (dba Verdun); Los Angeles (CA); Hybrid; USD 140,000 - 210,000; Full time

Job summary

Enari Inc. (dba Verdun) seeks an engineer to own an evaluation harness for multimodal security data, ensuring the system’s evaluation quality matches model quality across diverse real-world footage.

You will design replayable pipelines, calibrate confidence, and build an active-learning loop from disagreements and operator overrides. The role emphasizes production-grade Python, data tooling, notebooks, and eval services, with a focus on rapid iteration for prompts and models in a

Qualifications

• Experience shipping applied ML or LLM systems where eval quality mattered as much as model quality.
• Strong experiment design, error analysis, and offline/online benchmarking skills.
• Strong statistical instincts around calibration, uncertainty, and asymmetric risk.

Responsibilities

• Ownership of a replayable evaluation harness for multimodal security incidents.
• Develop golden sets and sliced evals across environmental conditions and camera types.
• Maintain operator-parity benchmarks with recall on true threats as the primary metric.

Skills

Experiment design

Error analysis

Benchmarking

Python

Data tooling

Notebooks

Eval services

Multimodal models

Prompt iteration

Job description

Own the evaluation harness that keeps our autonomy honest on messy real-world footage - where eval quality matters as much as model quality.

01 What you will own

• Replayable evaluation harness for multimodal security incidents


• Golden set and sliced evals across night, rain, low resolution, occlusion, and camera type


• Operator-parity benchmark, with recall on true threats as the primary metric


• Regression gates in CI for prompt, model, and provider changes


• Confidence calibration and abstention thresholds for autonomy decisions


• Active-learning loop from disagreements, false positives, false negatives, low-confidence cases, and operator overrides


• Benchmarking across quality, latency, and cost

02 What we are looking for

• Experience shipping applied ML or LLM systems where eval quality mattered as much as model quality


• Strong experiment design, error analysis, and offline / online benchmarking skills


• Strong statistical instincts around calibration, uncertainty, and asymmetric risk


• Production-quality Python and comfort with data tooling, notebooks, and eval services


• Comfort working with multimodal models, structured outputs, and rapid model / prompt iteration

03 Especially valuable

• Conformal prediction or calibration work


• Vision-language model eval or multi-frame video understanding


• Safety-critical ML or high-reliability systems


• Eval-platform or benchmark infrastructure


• Physical security, robotics, industrial vision, or adjacent real-world perception systems

04 Stack and architecture

• Multi-stage cascade from cheap first-pass filters to stronger multimodal reasoning, with human fallback for the tail


• Hybrid CV + VLM grounding, temporal reasoning over keyframes, and per-site context injection

Let’s build it together.