AI Assurance Engineer - Mission-Critical Systems

The Johns Hopkins University Applied Physics Laboratory; Laurel (MD); On-site; USD 100,000 - 245,000; Full time

Job summary

The Johns Hopkins University Applied Physics Laboratory (APL) is seeking an AI Assurance Engineer to design and execute rigorous evaluations of generative AI models and agentic systems for defense and mission-critical use cases.

You will develop test strategies, assess performance, security, and robustness, and collaborate with researchers and mission partners to enable safe deployment and trusted AI adoption.

Qualifications

• Bachelor’s degree in CS, AI, ML, data science, statistics or cybersecurity.
• Experience with Python and software development for AI, data analysis, or test automation.
• Familiarity with generative AI models, agentic systems, or ML concepts and AI threat landscape.
• Knowledge of test and evaluation principles, including test cases and success criteria.
• Strong analytical, written and verbal communication skills in multidisciplinary teams.
• Eligibility for Interim Secret clearance with U.S. citizenship.

Responsibilities

• Design and execute assurance evaluations for generative AI models and agentic systems supporting defense and healthcare missions.
• Develop test strategies, evaluation plans, test cases, and success criteria to assess AI performance, security, robustness, and mission suitability.
• Apply evaluation methods, benchmarks, and frameworks to identify model limitations and assurance risks.
• Collaborate with AI researchers, engineers, and mission partners to advance AI assurance capabilities.
• Analyze evaluation results and communicate findings, recommendations and assurance decisions to sponsors.
• Evaluate AI capabilities in realistic mission environments to enable risk-based deployment decisions.

Skills

Python

AI concepts

Test & evaluation

Communication

Security clearance readiness

Education

Bachelor's degree in Computer Science, Data Science, AI, ML, Statistics, Cybersecurity

Master's degree or higher (nice to have)

Job description

The Johns Hopkins University Applied Physics Laboratory (APL) is seeking an AI Assurance Engineer to design and execute rigorous evaluations of generative AI models and agentic systems for defense and mission-critical use cases.

You will develop test strategies, assess performance, security, and robustness, and collaborate with researchers and mission partners to enable safe deployment and trusted AI adoption.