Senior Software Engineer - AI Evaluation & System Design

CodeGeniusRecruit; California Hot Springs (CA); On-site; USD 120,000 - 200,000; Full time

Job summary

CodeGeniusRecruit is seeking a senior engineering-focused professional to vet knowledge work tasks and model outputs, and to write high-quality instruction specs. You will design challenging engineering tasks and evaluation sets, translating tacit judgment into teachable criteria for client researchers.

The role requires hands-on work with large language models in professional settings, a strong track record of shipping production software, and clear progression toward senior ownership of

Qualifications

• Bachelor's degree or higher in computer science, software engineering, or a related discipline.
• Strong relevant experience in building and shipping production software or engineered systems.
• Genuine specialization in at least one engineering area.
• Clear progression to a senior level with real ownership of systems and design decisions.
• Hands-on working use of large language models in professional work.

Responsibilities

• Vet the quality of engineering knowledge work tasks and model outputs.
• Write high-quality instruction specs and produce golden solutions to engineering problems.
• Design challenging engineering tasks and evaluation sets.
• Work with client researchers to maintain consistent standards.
• Translate tacit engineering judgment into explicit, teachable criteria.

Skills

Software engineering

LLMs experience

Leadership

Problem solving

Criteria design

Education

Bachelor's degree in CS/SE or related

Job description

CodeGeniusRecruit is seeking a senior engineering-focused professional to vet knowledge work tasks and model outputs, and to write high-quality instruction specs. You will design challenging engineering tasks and evaluation sets, translating tacit judgment into teachable criteria for client researchers.

The role requires hands-on work with large language models in professional settings, a strong track record of shipping production software, and clear progression toward senior ownership of