Staff AI Test and Evaluation Engineer

Anduril Industries, Inc.; Washington (District of Columbia); Hybrid; USD 253,000 - 336,000; Full time

Benefits offered by this job

Equity grants

Comprehensive health benefits

Job summary

Anduril Industries in Washington, DC is seeking a Discovery engineer to build evaluation pipelines for agentic AI and ML models, with focus on simulation environments and edge/classified platforms. You will own testing, metrics, and deployment readiness across complex systems.

The role emphasizes hands-on development, cross-functional collaboration, and delivering reliable evaluation infrastructure to support mission-critical AI capabilities.

Qualifications

• Technical Expertise: Bachelor’s or Master’s in CS/CS-related field.
• Programming Proficiency: 12+ years of production-grade coding, strong Python skills.
• Evaluation Ownership: experience owning evaluation or automated testing for ML/software.
• AI/ML Evaluation: designing evaluation methodologies, metrics, benchmarks.
• Simulation Experience: designing or using simulation environments for model testing.
• Metrics Discipline: defining and tracking performance metrics over time.
• Program Ownership: coordinating across teams, documenting results.
• Security Clearance: eligible for US security clearance.

Responsibilities

• Develop and maintain evaluation frameworks and pipelines for AI/ML models.
• Lead validation of agentic AI systems on classified platforms and edge hardware.
• Build reusable evaluation pipelines, test harnesses, and dashboards for monitoring.
• Define metrics and establish historic performance capture across releases.
• Collaborate with cross-functional teams to document test results and requirements.
• Own end-to-end evaluation infrastructure from unit checks to full-system assessments.
• Troubleshoot and debug issues uncovered in evaluation and deployment.

Skills

Python

Evaluation ownership

AI/ML evaluation

Simulation

Metrics discipline

Systems thinking

Program ownership

Security clearance

Education

CS/EE degree

Tools

Go

C++

Scripting

Job description

Washington, District of Columbia, United States

Anduril Industries is a defense technology company with a mission to transform U.S. and allied military capabilities with advanced technology. By bringing the expertise, technology, and business model of the 21st century’s most innovative companies to the defense industry, Anduril is changing how military systems are designed, built and sold. Anduril’s family of systems is powered by Lattice OS, an AI-powered operating system that turns thousands of data streams into a realtime, 3D command and control center. As the world enters an era of strategic competition, Anduril is committed to bringing cutting-edge autonomy, AI, computer vision, sensor fusion, and networking technology to the military in months, not years.

ABOUT THE TEAM

Discovery is Anduril's team for taking the newest problems across domains- space, missile systems, air, sensor capability,autonomy, and cyber- and proving out what is worth solving. We build the models, run the tests, and carry what works to thepoint where a program can pick it up. Discovery works alongside Air Defense, Space, Intelligence, Cyber, GNC, Hardware, andevery other group at Anduril to incubate the solutions to the hardest problems.

ABOUT THE JOB

Discovery is building an expeditionary force: a team of engineers who want to solve difficult problems and who navigate unfamiliar territory as an operating standard. As a Discovery engineer, you pick up a concept nobody has proven and build the analysis or prototype that tests it to get to an answer. Whether you spend your time working on hard problems on our autonomy stack, acoustic sensor analysis, or battle space management, we tackle every challenge the same way: by questioning assumptions and building our way to the answer.

We are focused on taking AI models from research into production—onto classified platforms and edge hardware where theyhave to perform reliably in the real world. As our model portfolio and classified work grow, rigorous, repeatable evaluation ofhow these models actually perform has become mission-critical.

WHAT YOU'LL DO

• Develop Test Scenarios Simulation Environments: Develop comprehensive test scenarios, and simulation environments to assess agentic AI performance in classified simulations.
• Validate AI on Classified & Edge Platforms: Lead the integration and validation of agentic AI systems onto classified platforms and edge hardware.
• Build Reusable Evaluation Pipelines: Build reusable evaluation pipelines, automated test harnesses, and monitoring dashboards for continuous validation.
• Define Actionable Metrics: Figure out what good metrics look like for AI models and traditional models—establishing systematic, historic capture of performance rather than one-shot, deployment-specific measurement.
• Partner Cross-Functionally: Partner with cross-functional teams to define requirements, document test results, and contribute to AI model cards and deployment readiness reviews.
• Own End-to-End Evaluation: Architect, build, and maintain the evaluation and validation infrastructure for our agentic AI and ML models, from unit-level model checks through full-system, scenario-based assessment.
• Troubleshoot and Debug: Analyze and resolve issues uncovered in evaluation and in deployment, ensuring reliability and operational success across every release.

REQUIRED QUALIFICATIONS

• Technical Expertise: Bachelor's or Master's degree in Computer Science, Machine Learning, Software Engineering, Computer Engineering, Electrical Engineering, or a related technical field.
• Programming Proficiency: At least 12+ years of hands-on experience writing production-grade code, with strong Python skills for building evaluation pipelines, test harnesses, and tooling.
• Evaluation Ownership Mindset: Demonstrated experience owning evaluation or automated testing for production ML or software systems—test scenarios, metrics, regression suites, and CI infrastructure.
• AI/ML Evaluation Experience: Hands-on experience designing evaluation methodologies for AI or machine learning models—defining metrics, building benchmarks, and assessing model behavior against real-world scenarios.
• Simulation Experience: Experience designing or working extensively with simulation environments to exercise model and system behavior.
• Metrics Discipline: Working knowledge of how to define, capture, and reason about performance metrics for AI models and traditional models, including systematic historic capture over time.
• Systems-Level Thinking: Ability to navigate and contribute to complex systems and established codebases.
• Program Ownership: Comfort operating between technical program management and software engineering—defining requirements, coordinating across teams, and documenting results.
• Real-World Impact: Passion for building the evaluation infrastructure that proves AI works—directly influencing mission-critical outcomes.
• Security Clearance: Must be eligible for a US security clearance.

PREFERRED QUALIFICATIONS

• Prior Title Background: Experience as an ML Test & Evaluation Engineer, SDET for ML systems, ML/Evaluation Engineer, Simulation Engineer, or Technical Program Manager for AI/ML.
• Agentic AI Evaluation: Experience designing test methodologies for agentic AI systems—tasking, decision‑making, and scenario‑based behavior validation.
• Classified / Edge Deployment: Familiarity with validating and deploying AI models onto classified platforms, edge hardware, or resource‑constrained environments.
• Model Cards & Readiness Reviews: Experience contributing to AI model cards, deployment readiness reviews, or similar model‑governance and release‑gating practices.
• Aerospace/Defense T&E: Familiarity with test and evaluation practices in aerospace or defense, including qualification testing, range operations, or operational assessment.
• MLOps & Monitoring: Experience with MLOps tooling, monitoring dashboards, and continuous validation pipelines for ML models in production.
• Programming Skills: Additional experience with Go, C++, or scripting for test automation and tooling.
• Growth into Broader Ownership: Interest in growing from T&E ownership into deeper ownership of the AI evaluation and deployment stack over time.

US Salary Range

$253,000 - $336,000 USD

The salary range for this role is an estimate based on a wide range of compensation factors, inclusive of base salary only. Actual salary offer may vary based on (but not limited to) work experience, education and/or training, critical skills, and/or business considerations. Highly competitive equity grants are included in the majority of full time offers; and are considered part of Anduril's total compensation package. Additionally, Anduril offers top‑tier benefits for full‑time employees, including:

At Anduril, we invest in our people. Our comprehensive, competitive benefits package (available at little to no cost to employees) ensures you’re supported in health, recovery, and whatever comes next. For more information, Explore Our Benefits .

Voluntary Self-Identification

For government reporting purposes, we ask candidates to respond to the below self‑identification survey.Completion of the form is entirely voluntary. Whatever your decision, it will not be considered in the hiringprocess or thereafter. Any information that you do provide will be recorded and maintained in aconfidential file.

As set forth in Anduril Industries’s Equal Employment Opportunity policy,we do not discriminate on the basis of any protected group status under any applicable law.

If you believe you belong to any of the categories of protected veterans listed below, please indicate by making the appropriate selection.As a government contractor subject to the Vietnam Era Veterans Readjustment Assistance Act (VEVRAA), we request this information in order to measurethe effectiveness of the outreach and positive recruitment efforts we undertake pursuant to VEVRAA. Classification of protected categoriesis as follows:

A \"disabled veteran\" is one of the following: a veteran of the U.S. military, ground, naval or air service who is entitled to compensation (or who but for the receipt of military retired pay would be entitled to compensation) under laws administered by the Secretary of Veterans Affairs; or a person who was discharged or released from active duty because of a service‑connected disability.

A \"recently separated veteran\" means any veteran during the three‑year period beginning on the date of such veteran's discharge or release from active duty in the U.S. military, ground, naval, or air service.

An \"active duty wartime or campaign badge veteran\" means a veteran who served on active duty in the U.S. military, ground, naval or air service during a war, or in a campaign or expedition for which a campaign badge has been authorized under the laws administered by the Department of Defense.

An \"Armed forces service medal veteran\" means a veteran who, while serving on active duty in the U.S. military, ground, naval or air service, participated in a United States military operation for which an Armed Forces service medal was awarded pursuant to Executive Order 12985.