AI Benchmarking Scientist - Onsite Ft. Meade (TS/SCI)

Precision Solutions, Inc.; Fort Meade (MD); On-site; USD 100,000 - 130,000; Full time

Job summary

Precision Solutions, Inc. is seeking a Data Scientist in Fort Meade, MD, focusing on designing and maintaining AI benchmarks using InspectAI. Candidates must hold an active TS/SCI with FS Poly clearance. Responsibilities include developing evaluation datasets and metrics to assess model performance.

Ideal applicants will have a BS in Computer Science or similar, alongside skills in Python, MongoDB, and statistical analysis. Collaborative tools like Jira or Confluence and experience with data validation are also essential.

This position is onsite and particularly suitable for those with expertise in working in constrained environments.

Qualifications

• Experience with AI Evaluation & Frameworks such as InspectAI and LLM Benchmarking.
• Experience with Statistical Analysis, Data Cleaning & Validation.
• Familiarity with Git Version Control, Dataset Lineage/Traceability.

Responsibilities

• Design and maintain benchmarks for model testing and workflow testing using InspectAI.
• Clean, transform, validate, and analyze supporting data.
• Document dataset lineage and evaluation outputs for reproducibility.

Skills

InspectAI

Python

MongoDB

Jupyter notebooks

Data visualization tools

Statistical analysis

Data cleaning and validation

Cloud or containerized environments

Git-based version control

Collaboration tools such as Jira or Confluence

Education

BS in Computer Science or similar technical field

Job description

Precision Solutions, Inc. is seeking a Data Scientist in Fort Meade, MD, focusing on designing and maintaining AI benchmarks using InspectAI. Candidates must hold an active TS/SCI with FS Poly clearance. Responsibilities include developing evaluation datasets and metrics to assess model performance.

Ideal applicants will have a BS in Computer Science or similar, alongside skills in Python, MongoDB, and statistical analysis. Collaborative tools like Jira or Confluence and experience with data validation are also essential.

This position is onsite and particularly suitable for those with expertise in working in constrained environments.