Senior Engineering Domain Expert - GenAI Benchmarks

Mercor; United States; Hybrid; USD 180,000 - 240,000; Full time

Job summary

Cincinnatus LLC is hiring a senior engineering and software domain expert to work directly with a leading AI lab's research and program management teams. You will review engineering knowledge work tasks, write instruction specs, and build benchmarks that measure model improvements.

This is a full-time W-2 with on-site Bay Area work requirements. You will review task quality, craft golden datasets, and define engineering tasks reflecting real-world practice, with relocation to the Bay Area as

Qualifications

• Bachelor's degree or higher in computer science, software engineering, or a closely related engineering discipline.
• An MS or PhD is strongly preferred.
• Genuine specialization in at least one area such as distributed systems, security, ML systems, or quality engineering.

Responsibilities

• Data QA and reviews to vet engineering knowledge work tasks and model outputs.
• Write instruction specs and golden datasets for engineering tasks.
• Design benchmarks and domain depth for engineering tasks and evaluation sets.
• Calibrate standards with client researchers and translate tacit judgment into explicit criteria.

Skills

AI fluency

Distributed systems

Code quality & reviews

Documentation & specs

Education

Bachelor's degree in Computer Science or related field

MS or PhD preferred

Job description

Cincinnatus LLC is hiring a senior engineering and software domain expert to work directly with a leading AI lab's research and program management teams. You will review engineering knowledge work tasks, write instruction specs, and build benchmarks that measure model improvements.

This is a full-time W-2 with on-site Bay Area work requirements. You will review task quality, craft golden datasets, and define engineering tasks reflecting real-world practice, with relocation to the Bay Area as