Senior AI Engineering Specialist — GenAI Benchmarks

Mercor; San Francisco (CA); Hybrid; USD 180,000 - 240,000; Full time

Job summary

Cincinnatus LLC is hiring a senior engineering and software domain expert to work with a leading AI lab, focusing on evaluating engineering knowledge work, writing instruction specs, and building benchmarks. This is a full-time W-2 role with the option to be placed at the client's AI lab, hybrid in the Bay Area, on-site several days per week.

Relocation at your cost is required if needed. You should have a BS or higher in CS or related field (MS/PhD preferred), 4+ years of production software

Qualifications

• Bachelor's degree or higher in CS or related engineering field; MS/PhD preferred.
• 4+ years of substantive professional engineering practice shipping production software.
• Genuine specialization in at least one domain (distributed systems, ML infra, security/cryptography, embedded, or developer tooling).
• Senior-level progression with ownership of systems and design decisions.
• Hands-on experience with large language models in professional work.

Responsibilities

• Vet the quality of engineering knowledge work tasks and model outputs.
• Write instruction specs and golden solutions for engineering problems.
• Design benchmarks and evaluation sets for engineering tasks.
• Calibrate standards with client researchers to translate tacit judgment into explicit criteria.

Skills

Software engineering

Backend systems

AI fluency

Written communication

Availability

Education

Bachelor's degree or higher in CS or related field

MS or PhD preferred

Job description

Cincinnatus LLC is hiring a senior engineering and software domain expert to work with a leading AI lab, focusing on evaluating engineering knowledge work, writing instruction specs, and building benchmarks. This is a full-time W-2 role with the option to be placed at the client's AI lab, hybrid in the Bay Area, on-site several days per week.

Relocation at your cost is required if needed. You should have a BS or higher in CS or related field (MS/PhD preferred), 4+ years of production software