AI/ML Test Engineer

Modern Technology Solutions, Inc. (MTSI); Springfield (VA); On-site; USD 90,000 - 120,000; Full time

Job summary

Modern Technology Solutions, Inc. (MTSI) in Springfield, VA is seeking an experienced AI/ML Test Engineer to lead system-level evaluations in challenging environments. Candidates must possess a TS/SCI clearance and CI Polygraph.

Responsibilities include executing AI/Machine Learning evaluations, integration testing, and supporting automated evaluation pipelines. The ideal candidate will have a strong background in operational testing and excellent analytical skills.

Qualifications

• 4+ years of experience supporting AI/ML testing and evaluation workflows.
• Familiarity with automated testing pipelines and model performance assessment concepts.
• Ability to support multiple evaluation activities across dynamic environments.

Responsibilities

• Execute system-level AI/Machine Learning evaluation activities.
• Perform integration testing and data capture across AI/ML evaluation workflows.
• Support automated evaluation pipelines for testing and performance assessment.

Skills

AI/ML testing

System integration

Operational testing

Troubleshooting technical workflows

Analytical skills

Communication skills

Education

Bachelor’s degree in IT or STEM

Job description

Modern Technology Solutions, Inc. (MTSI) is hiring an AI/ML Test Engineer in Springfield, VA. Qualified candidates must be U.S. citizens with a TS/SCI clearance and CI Polygraph.

Responsibilities

• Execute system-level AI/Machine Learning evaluation activities within the Test & Evaluation (T&E) environment supporting mission-focused operational testing.
• Perform integration testing, data capture, drift detection, and operational validation activities across AI/ML evaluation workflows.
• Support automated and semi-automated evaluation pipelines for AI/ML model testing and performance assessment.
• Develop, maintain, and refine data annotation workflows supporting operational evaluation requirements.
• Validate operational datasets and support testing activities across multiple models, datasets, and mission scenarios.
• Conduct integration testing of AI/ML models within operationally representative environments and workflows.
• Adapt evaluation procedures and workflows to address evolving GEOINT data conditions and operational mission requirements.
• Support high-throughput and parallel model evaluation activities to maintain alignment with mission timelines and operational priorities.
• Identify and mitigate data drift, performance degradation, and workflow issues impacting evaluation reliability.
• Collaborate with Senior AI/ML Test Engineers, data scientists, software engineers, and mission stakeholders to support scalable T&E execution.
• Contribute to technical documentation, evaluation reporting, and customer briefings supporting operational decision-making.

Qualifications Required

• 4+ years of experience supporting AI/ML testing, system integration, operational testing, or evaluation workflows.
• Familiarity with automated testing pipelines, operational data validation, and model performance assessment concepts.
• Ability to support multiple evaluation activities across dynamic operational environments.
• Experience troubleshooting technical workflow or data-related issues in support of operational objectives.
• Strong analytical, organizational, and communication skills.

Education Requirements

• Bachelor’s degree in IT or STEM related field, or additional relevant experience in lieu of a degree.

Desired Qualifications

• Experience supporting DoD, NGA, or Intelligence Community programs.
• Familiarity with AI/ML model evaluation, GEOINT workflows, and operational testing environments.
• Experience working in Agile, DevSecOps, or cloud-based operational environments.
• Familiarity with annotation workflows, data capture pipelines, drift detection, and automated evaluation tools.
• Experience supporting scalable AI/ML evaluation or mission-focused T&E activities.

Clearance Requirements

• Active TS/SCI clearance and the ability to pass a CI polygraph within 30 days.