Agentic AI Evaluation Engineer (Edge & Classified)

Anduril Industries; Costa Mesa (CA); On-site; USD 253,000 - 336,000; Full time

Job summary

Anduril Industries is seeking a Discovery engineer to build and validate evaluation infrastructure for agentic AI and ML models across classified platforms and edge hardware.

You will design test scenarios, develop simulation environments, and establish metrics and pipelines to ensure reliable performance in mission-critical settings, partnering with cross-functional teams and owning end-to-end evaluation.

Qualifications

• Bachelor's or Master's degree in Computer Science, Machine Learning, Software Engineering, Computer Engineering, Electrical Engineering, or a related technical field.
• 8-10 years of hands-on experience writing production-grade code, with strong Python skills for building evaluation pipelines, test harnesses, and tooling.
• Experience owning evaluation or automated testing for production ML or software systems—test scenarios, metrics, regression suites, and CI infrastructure.
• Hands-on experience designing evaluation methodologies for AI or machine learning models—defining metrics, building benchmarks, and assessing model behavior against real-world scenarios.
• Experience designing or working extensively with simulation environments to exercise model and system behavior.
• Knowledge of defining, capturing, and reasoning about performance metrics for AI models and traditional models, including historic capture over time.
• Ability to navigate complex systems and established codebases.
• Comfort operating between technical program management and software engineering—defining requirements, coordinating across teams.
• Passion for building evaluation infrastructure that proves AI works in mission-critical contexts.
• Must have a TS/SCI clearance.

Responsibilities

• Architect, build, and maintain the evaluation and validation infrastructure for our agentic AI and ML models, from unit-level checks to full-system scenario-based assessment.
• Develop test scenarios and simulation environments to assess agentic AI performance in classified simulations.
• Lead integration and validation of agentic AI systems onto classified platforms and edge hardware.
• Build reusable evaluation pipelines, automated test harnesses, and monitoring dashboards for continuous validation.
• Define actionable metrics and establish historic capture of performance over time.
• Partner with cross-functional teams to define requirements and document test results.
• Troubleshoot and debug issues uncovered in evaluation and deployment, ensuring reliability across releases.

Skills

Python programming

Evaluation pipelines

Test harnesses

CI infrastructure

Education

Bachelor's or Master's in Computer Science / ML / Software / CE / EE

Tools

Go

C++

Scripting

Job description

Anduril Industries is seeking a Discovery engineer to build and validate evaluation infrastructure for agentic AI and ML models across classified platforms and edge hardware.

You will design test scenarios, develop simulation environments, and establish metrics and pipelines to ensure reliable performance in mission-critical settings, partnering with cross-functional teams and owning end-to-end evaluation.