Postdoc: Human-Subjects AI Benchmarking

University of Virginia; Charlottesville (VA); On-site; USD 60,000 - 75,000; Full time

Benefits offered by this job

UVA Health Plan

Vision coverage

Dental plan

Life insurance

Disability benefits

Paid time off

Job summary

University of Virginia invites applications for a research position to design, execute, and analyze a large paired human-subjects study linking AI biosecurity benchmarks to protocol-design tasks. The role reports to the Associate Professor and collaborates with cross-disciplinary investigators and a lab partner.

The candidate will contribute to a parallel machine unlearning effort, manage participant data, and help prepare manuscripts with open releases while ensuring IRB compliance.

Qualifications

• PhD in data science, statistics, biostatistics, psychometrics, or related field completed by the time of hire.
• Experience designing and analyzing human subjects studies, including power analysis and study design.
• Strong statistical programming in R or Python, with mixed-effects or hierarchical modeling.
• Working knowledge of molecular biology or microbiology to evaluate protocols and collaborate with lab scientists.

Responsibilities

• Lead day-to-day execution of the paired evaluation study, including benchmark administration and participant scheduling.
• Develop scoring rubrics and instruments for protocol-design and troubleshooting tasks.
• Run mixed-effects analyses to relate benchmark scores to real-world task performance with uncertainty estimates.
• Produce the benchmark validity report and crosswalk/calibration framework.
• Contribute to machine unlearning and model editing analyses, including within-subject comparisons of model variants.
• Prepare manuscripts and documentation for open release and support IRB compliance.

Skills

Study design

Statistical programming

Power analysis

Education

PhD or equivalent in data science/biostatistics/psychometrics

Tools

PyTorch

Hugging Face

Job description

University of Virginia invites applications for a research position to design, execute, and analyze a large paired human-subjects study linking AI biosecurity benchmarks to protocol-design tasks. The role reports to the Associate Professor and collaborates with cross-disciplinary investigators and a lab partner.

The candidate will contribute to a parallel machine unlearning effort, manage participant data, and help prepare manuscripts with open releases while ensuring IRB compliance.