AI Software Engineer - Evaluation & Benchmarks

Hire Feed; San Francisco (CA); On-site; USD 120,000 - 190,000; Full time

Job summary

Hire Feed is seeking an Engineering & Software Domain Expert to help advance AI model development. The role involves reviewing engineering knowledge work tasks, writing instruction specs, and creating golden solutions that define what correct engineering looks like in real-world software.

This full-time position places you with a leading AI lab as part of their extended workforce, collaborating with research teams.

Qualifications

• Experience in software engineering with algorithms, data structures and system design.
• Ability to review code quality, identify edge cases and assess real-world engineering standards.
• Proficiency in writing clear instruction specifications and technical documentation.
• Experience designing benchmarks and evaluation frameworks for software development tasks.

Responsibilities

• Vet the quality of engineering knowledge work tasks and model outputs.
• Write high-quality instruction specifications and produce golden solutions.
• Design challenging engineering tasks and evaluation sets to assess model performance.
• Collaborate with research and program management teams to improve how frontier AI models reason about engineering and software development.
• Build engineering-specific tools alongside research teams using client-issued accounts and equipment.

Skills

Software engineering

Algorithms & DS

Code review

Instruction specs

Benchmark design

Problem solving

Job description

Hire Feed is seeking an Engineering & Software Domain Expert to help advance AI model development. The role involves reviewing engineering knowledge work tasks, writing instruction specs, and creating golden solutions that define what correct engineering looks like in real-world software.

This full-time position places you with a leading AI lab as part of their extended workforce, collaborating with research teams.