Senior AI Engineering Domain Expert — Data QA & Benchmarks

Dorado; California, Northern (MO, KY); Hybrid; USD 180,000 - 240,000; Full time

Job summary

Cincinnatus LLC is hiring a senior engineering and software domain expert to join a leading AI lab's GenAI team. You will review engineering knowledge work tasks, write instruction specs and golden solutions, and build benchmarks to measure model improvement.

This full-time on-site Bay Area role requires 40 hours per week for an initial 6-month engagement, with relocation to the Bay Area at your own cost if needed. Strong AI fluency is essential.

Qualifications

• Bachelor's degree or higher in computer science, software engineering, or a closely related engineering discipline.
• An MS or PhD is strongly preferred.
• 4+ years of substantive professional engineering practice, building and shipping production software or engineered systems.
• Genuine specialization in at least one area (distributed systems, backend, security/cryptography, ML systems, developer tooling, or quality/test engineering).
• Clear progression to a senior level (Senior, Staff, Principal, Architect, or Engineering Lead) with ownership of systems and design decisions.
• Hands-on with AI fluency and large language models in professional work.
• Able to commit reliably to 40 hours per week for an initial engagement of 6 months.
• On-site Bay Area, California; relocation at own cost if not currently based there.

Responsibilities

• Data QA and reviews: Vet quality of engineering knowledge work tasks and model outputs, spot missing behaviors and edge cases.
• Write instruction specs and golden datasets; define new engineering tasks reflecting real-world work.
• Design benchmarks and domain depth; collaborate with researchers to build engineering tools and skills.
• Calibration: align standards with client researchers; translate tacit judgments into explicit criteria.

Job description

Cincinnatus LLC is hiring a senior engineering and software domain expert to join a leading AI lab's GenAI team. You will review engineering knowledge work tasks, write instruction specs and golden solutions, and build benchmarks to measure model improvement.

This full-time on-site Bay Area role requires 40 hours per week for an initial 6-month engagement, with relocation to the Bay Area at your own cost if needed. Strong AI fluency is essential.