Senior AI Evaluation & Validation Lead

Leonardo DRS; Frederick (MD); Hybrid; USD 120,000 - 180,000; Full time

Benefits offered by this job

Health benefits

401(k)

4/10 schedule

Flexible schedule

Job summary

Leonardo DRS is seeking a seasoned data/ML evaluation engineer for AIS. You will design and validate AI capabilities, build pipelines, and ensure measurement integrity from ideation through deployment.

You’ll own the truth behind every claim and report results to Finance with rigorous controls. The role emphasizes end-to-end ownership within the AI Excellence function, with high visibility and cross-team collaboration.

Qualifications

• 5+ years in data, ML, or test engineering with 3+ years in evaluation or model validation, including LLM evaluation work.
• Proven experience in bi-directional evaluation rigor and showing impact on shipping decisions.
• Built a measurement practice with baselines, pipelines, and leadership dashboards.

Responsibilities

• Define “good enough to ship.” Design evaluation criteria, gate launches and document baselines and acceptance.
• Build the measurement pipelines and record-keeping for capability delivery and tracking.
• Establish the truth by linking measurements to source-of-record data and ground truth.
• Define deployed systems logging requirements for diagnosability and auditability.
• Produce monthly and quarterly results that finance can defend and forecast against actuals.
• Manage record intake to capture every AI idea and keep it on-books.

Skills

Data science

ML evaluation

Model validation

Dashboards

Education

Bachelor’s degree in Engineering/CS/DS

Tools

Pipelines

Dashboards

Job description

Leonardo DRS is seeking a seasoned data/ML evaluation engineer for AIS. You will design and validate AI capabilities, build pipelines, and ensure measurement integrity from ideation through deployment.

You’ll own the truth behind every claim and report results to Finance with rigorous controls. The role emphasizes end-to-end ownership within the AI Excellence function, with high visibility and cross-team collaboration.