Data Scientist

Precision Solutions Ag; Fort Meade (MD); On-site; USD 120,000 - 160,000; Full time

Job summary

Precision Solutions is seeking a Data Scientist to design, build, and maintain benchmarks for AI model testing and agentic workflows at Fort Meade, MD. The role focuses on evaluating model performance, data curation, and reproducible workflows using InspectAI and related tools.

The candidate will define evaluation datasets, metrics, and scoring methods, while collaborating with Researchers and Developers to operationalize benchmarks in constrained environments.

Qualifications

• Experience with AI Evaluation & Frameworks such as InspectAI, LLM Benchmarking, Agentic Workflow Evaluation (tool-use, multi-step task performance).
• Experience with Statistical Analysis, Data Cleaning & Validation, Data Visualization Tools, Jupyter Notebooks; Data & Environment: MongoDB, Cloud/Containerized Environments, Airgapped & Constrained Operations.
• Familiarity with Git Version Control, Experimental Design, Dataset Lineage/Traceability, Jira, and Confluence.
• BS in Computer Science or similar technical field; 4 years of relevant experience may substitute for degree.

Responsibilities

• Design, build, and maintain benchmarks for model testing and agentic workflow testing using InspectAI.
• Define evaluation datasets, test scenarios, metrics, scoring approaches, and repeatable evaluation workflows.
• Collaborate with Researchers to ground benchmark design and with Developers to operationalize evaluation workflows.
• Document dataset lineage, scoring rationale, benchmark assumptions, and evaluation outputs for reproducibility.

Skills

InspectAI

Python

MongoDB

Jupyter notebooks

Data visualization tools

Statistical analysis

Data cleaning and validation

Cloud or containerized environments

Git-based version control

Jira

Confluence

Airgapped environments

Education

BS in Computer Science or related field

Tools

Jira

Confluence

Git

Docker

Kubernetes

Job description

Overview

Data Scientist

Onsite | Ft. Meade, MD

Active TS/SCI w/FS Poly (NSA) Clearance Required

Summary

Since 2012, our client has helped mission-critical government organizations and businesses face their most daunting technology challenges. Their team have been trusted partners to many government agencies and are extremely familiar with a wide variety of systems, policies, and procedures. Our client is also a distinguished custom software development firm dedicated to delivering premium solutions tailored for businesses and governmental needs. They are home to top-tier technology professionals recognized as industry pioneers, comprehensive full-stack engineers, and reliable consultants. These experts are adept at clear communication, excel in resolving complex challenges where others may falter, and are skilled in actualizing an organization's vision.

The Data Scientists' primary responsibility is to design, build, and maintain benchmarks for model testing and agentic workflow testing using InspectAI. They define evaluation datasets, test scenarios, metrics, scoring approaches, and repeatable evaluation workflows that measure model and agent performance in a consistent, auditable way. They also clean, transform, validate, and analyze supporting data; generate metrics and visualizations; and produce findings that support model evaluation, reporting, and decision-making. They partner with Researchers to ground benchmark design in sound methodology, and with Developers to operationalize evaluation workflows into stable, repeatable tooling. Data Scientists should also be able to document dataset lineage, scoring rationale, benchmark assumptions, and evaluation outputs so results are reproducible, traceable, and usable in constrained or airgapped environments.

Upcoming Tasking

• AI Model Benchmarks - Design, implement, and maintain benchmark suites that measure model performance against defined evaluation criteria, including dataset curation, scoring, and results reporting.
• Agentic AI Benchmarks - Develop benchmarks and test scenarios that evaluate agentic workflows and multi-step task performance, including tool-use, task completion, and reliability metrics.
• Evaluation Dataset Development - Curate, validate, and version evaluation datasets and test scenarios, including any adversarial or edge-case sets needed to stress agent behavior.
• Metrics, Analysis, and Reporting - Produce scoring frameworks, visualizations, and written analyses that translate raw evaluation results into actionable findings for stakeholders.

Technologies/Skills Needed

• InspectAI
• Python
• MongoDB
• Jupyter notebooks
• Data visualization tools
• Statistical analysis
• Data cleaning and validation
• Cloud or containerized environments
• Git-based version control
• Collaboration tools such as Jira or Confluence
• Experience operating in airgapped or constrained environments

Requirements

• You have experience with AI Evaluation & Frameworks such as InspectAI, LLM Benchmarking, Agentic Workflow Evaluation (tool-use, multi-step task performance)
• You have experience with Statistical Analysis, Data Cleaning & Validation, Data Visualization Tools, Jupyter NotebooksData & Environment: MongoDB, Cloud/Containerized Environments, Airgapped & Constrained Operations
• You're familiar with Git Version Control, Experimental Design, Dataset Lineage/Traceability, Jira, and Confluence
• BS in Computer Science or similar technical field (4 years of relevant experience may be substituted in lieu of a BS degree)

Clearance Requirements

• Applicants selected will be subject to a security investigation and may need to meet eligibility requirements for access to classified information; an active TS/SCI w/FS Poly clearance is required. Please note that the FS Poly currently needs to be held by the NSA or within the past two years.

Other Duties

Please note this job description is not designed to cover or contain a comprehensive listing of activities, duties, or responsibilities that are required of the employee for this job. Duties, responsibilities, and activities may change at any time with or without notice.

About Us

Northern Virginia-based Precision Solutions is an expert in staffing solutions for companies of any size that open the door to new opportunities and seek outstanding talent. We pride ourselves on being versatile enough to tailor our relationships to the needs of each individual client, being agile in the fast-paced marketplace, and being precise in meeting the needs of any company.

Equal Opportunity Employer Statement

Precision Solutions is an equal opportunity employer. We prohibit discrimination and harassment of any kind based on race, color, sex, religion, sexual orientation, national origin, disability, genetic information, pregnancy, or any other protected characteristic as outlined by federal, state, or local laws.