National Security AI Scientist: Foundation Models & Agents

Pacific Northwest National Laboratory; Kennewick (WA); On-site; USD 120,000 - 180,000; Full time

Job summary

Pacific Northwest National Laboratory seeks a Data Scientist in the AI and Data Analytics Division to design and run evaluation studies for AI/ML systems and agents. You will build test data, prompts, and scenarios, and create reproducible workflows and benchmarks.

Responsibilities include executing controlled experiments, analyzing model behavior, and documenting methods and results to support mission-relevant criteria. Onsite work in Richland or Seattle, with security clearance requirements.

Qualifications

• Minimum qualifications include a BS/BA with 2 years data science experience or an MS/MA or PhD.
• Experience developing or evaluating LLMs, VLMs, pretrained transformer models, AI agents, or integrated AI systems.
• Experience using Python and machine-learning frameworks (e.g., TensorFlow, PyTorch).

Responsibilities

• Design and implement evaluation studies for AI/ML systems including pretrained models and AI agents.
• Prepare datasets, prompts, scenarios, baselines, and scoring rubrics for rigorous assessment.
• Develop evaluation environments, test harnesses, and agentic components to exercise models and workflows.
• Execute controlled experiments at model, component, workflow, and system levels.
• Analyze model behavior to identify capabilities, weaknesses, and data sensitivity.
• Document methods, results, uncertainties, and limitations in reproducible code and reports.

Skills

LLMs evaluation

AI agents integration

Data analysis

Education

BS/BA in related field

MS/MA in related field

PhD

Tools

Python

TensorFlow/PyTorch

LangGraph

Job description

Pacific Northwest National Laboratory seeks a Data Scientist in the AI and Data Analytics Division to design and run evaluation studies for AI/ML systems and agents. You will build test data, prompts, and scenarios, and create reproducible workflows and benchmarks.

Responsibilities include executing controlled experiments, analyzing model behavior, and documenting methods and results to support mission-relevant criteria. Onsite work in Richland or Seattle, with security clearance requirements.