GenAI Evaluation Scientist - PREP Research

Johns Hopkins University; Gaithersburg (MD); On-site; USD 70,000 - 110,000; Full time

Job summary

Johns Hopkins University in Maryland seeks a PREP Research Associate to support the NIST GenAI program, providing development support for AI evaluation platforms on a GPU-accelerated cluster. You will design and implement evaluation infrastructures, build data schemas, and contribute to reports aligning with program objectives, including work on deepfakes and agentic evaluation.

This role offers cutting-edge research exposure and collaboration with national labs and universities under PREP, with

Qualifications

• Proficiency in a mainstream programming language used in scientific and networking domains (Python, Java, JavaScript, Ruby, R, C, C++, Rust).
• Basic proficiency with networking technologies and REST-based applications.
• Proficiency with shell scripting (Bash, Zsh, etc.).
• Familiarity with virtualization and cloud tech stacks (Docker, Kubernetes, AWS).
• Knowledge of Computer Vision/Signal Processing fundamentals.

Responsibilities

• Installing, developing and deploying software for executing AI and ML systems on a GPU-accelerated cluster.
• Conducting image/audio/video evaluation series for advancing generative AI technologies.
• Experimental design and dataset curation.
• Developing baseline algorithms in generative AI.
• Upgrading and maintaining NIST GenAI software and evaluation infrastructure.
• Managing data formats and translations.
• Designing agentic evaluation metrics and frameworks for multi-agent systems.

Skills

Python

Java

JavaScript

Ruby

R

C

C++

Rust

Tools

Docker

Kubernetes

AWS

REST APIs

Job description

Johns Hopkins University in Maryland seeks a PREP Research Associate to support the NIST GenAI program, providing development support for AI evaluation platforms on a GPU-accelerated cluster. You will design and implement evaluation infrastructures, build data schemas, and contribute to reports aligning with program objectives, including work on deepfakes and agentic evaluation.

This role offers cutting-edge research exposure and collaboration with national labs and universities under PREP, with