GenAI Systems Engineer (PhD) — Data QA & Benchmarks

Mercor; San Francisco (CA); Hybrid; USD 180,000 - 240,000; Full time

Job summary

Cincinnatus LLC is hiring a senior engineering and software domain expert to work with a leading AI lab's research and program management teams. You will review engineering knowledge work, write instruction specs and golden solutions, and build benchmarks to evaluate model performance.

The role is hybrid and based in the Bay Area, California. You must live in the Bay Area or relocate at your own cost; relocation support is not provided.

Qualifications

• Bachelor's degree or higher in computer science, software engineering, or a closely related engineering discipline. An MS or PhD is strongly preferred.
• 4+ years of substantive professional engineering practice, building and shipping production software or engineered systems.
• Genuine specialization in at least one area such as distributed systems, security/cryptography, embedded/firmware, ML systems, developer tooling, or quality/test engineering.
• Clear progression to a senior level (Senior, Staff, Principal, Architect, or Engineering Lead) with ownership of systems and design decisions.
• Hands-on use of large language models in professional work and ability to distinguish well-reasoned from plausible-sounding but wrong answers.
• W-2 employment with ability to work on-site in the Bay Area, CA; hybrid role with on-site presence required.

Responsibilities

• Data QA and reviews: Vet the quality of engineering knowledge work tasks and model outputs.
• Instruction specs and golden datasets: Write specs and golden solutions for engineering problems.
• Benchmarks and domain depth: Design challenging tasks and evaluation sets for engineers.
• Calibration: Align standards with client researchers and translate tacit judgment into explicit criteria.

Skills

Engineering domain expertise

Senior-level ownership

AI fluency

Excellent written communication

4+ years professional engineering

Education

Bachelor's or higher in CS/SE

MS or PhD preferred

Job description

Cincinnatus LLC is hiring a senior engineering and software domain expert to work with a leading AI lab's research and program management teams. You will review engineering knowledge work, write instruction specs and golden solutions, and build benchmarks to evaluate model performance.

The role is hybrid and based in the Bay Area, California. You must live in the Bay Area or relocate at your own cost; relocation support is not provided.