GenAI Benchmark Research Scientist — Remote, Part-Time

Obsidian; New York (NY); Remote; USD 120,000 - 150,000; Full time

Job summary

Cincinnatus LLC is seeking experienced ML practitioners to author complex, multi-step ML tasks for frontier models. You will design experiments, implement changes, run trainings, and analyze results to determine where models fall short.

This full-time, fully remote role within the United States requires a MSc/PhD or equivalent, 1+ years research experience, and strong Python/Git skills, with a focus on RL and LLM evaluation.

Qualifications

• MSc or PhD in ML, CS, or equivalent research experience.
• 1+ years in a research or research-engineering role.
• Hands-on ML model training and evaluation end-to-end.
• Strong familiarity with large language models and evaluation techniques.
• Proficient in Python and Git; comfortable with scripting and notebooks.
• Understanding of reinforcement learning concepts preferred.
• Experience in AI training, model evaluation, or benchmark authoring preferred.
• Detail-oriented, creative task design, strong written communication, independent work.
• Availability for ~35 hours per week.

Responsibilities

• Design tasks from real ML research ideas into defined multi-step tasks.
• Run experiments: implement changes, train, analyze results.
• Explore RL ideas around reward functions and training behavior.
• Evaluate frontier models and identify gaps.
• Collaborate with researchers to keep tasks rigorous and fair.

Skills

LLM knowledge

Experimentation experience

Reinforcement learning basics

Python & Git

Communication skills

Independent worker

Detail-oriented

Education

MSc or PhD in ML/CS

Tools

Python

Git

Job description

Cincinnatus LLC is seeking experienced ML practitioners to author complex, multi-step ML tasks for frontier models. You will design experiments, implement changes, run trainings, and analyze results to determine where models fall short.

This full-time, fully remote role within the United States requires a MSc/PhD or equivalent, 1+ years research experience, and strong Python/Git skills, with a focus on RL and LLM evaluation.