Agentic AI Evaluation Engineer — Classified & Edge

Anduril Industries; Costa Mesa (CA); On-site; USD 253,000 - 336,000; Full time

Job summary

Anduril Industries in the United States is seeking a Discovery engineer to build and operate evaluation frameworks for agentic AI and ML models. You will design test scenarios, validation pipelines, and metrics for performance on classified simulations and edge hardware.

You will partner with cross-functional teams, own end-to-end evaluation infrastructure, and drive rigorous testing to ensure reliability and mission impact across all releases.

Qualifications

• Bachelor’s or Master’s degree in Computer Science, Machine Learning, Software/Computer Engineering, Electrical Engineering, or related field.
• 8–10 years hands-on production code experience, strong Python for eval pipelines and tooling.
• Experience owning evaluation or automated testing for production ML/software systems.
• Hands-on experience designing evaluation methodologies for AI/ML models.
• Experience with simulation environments to exercise model behavior.
• Strong discipline in defining and capturing performance metrics over time
• Comfort navigating between technical program management and software engineering.

Responsibilities

• Develop test scenarios and simulation environments to assess agentic AI performance in classified simulations.
• Validate AI on classified and edge platforms, including integration onto edge hardware.
• Build reusable evaluation pipelines, automated test harnesses, and dashboards for continuous validation.
• Define actionable metrics and establish historic performance capture across releases.
• Collaborate with cross-functional teams to document results and contribute to deployment readiness reviews.
• Own end-to-end evaluation infrastructure for agentic AI/ML models from unit checks to full-system assessments.
• Troubleshoot issues uncovered during evaluation or deployment to ensure reliability.

Skills

TS/SCI clearance

Python

Evaluation pipelines

CI infrastructure

Simulation design

Metrics engineering

Systems thinking

Program ownership

Real-world impact

Defense domain

Education

Advanced degree in CS/ML/EE

Tools

Go

C++

Scripting

Job description

Anduril Industries in the United States is seeking a Discovery engineer to build and operate evaluation frameworks for agentic AI and ML models. You will design test scenarios, validation pipelines, and metrics for performance on classified simulations and edge hardware.

You will partner with cross-functional teams, own end-to-end evaluation infrastructure, and drive rigorous testing to ensure reliability and mission impact across all releases.