Research Engineer — AI Evaluation & Release Readiness

AGI Inc; San Francisco (CA); On-site; USD 120,000 - 160,000; Full time

Benefits offered by this job

Competitive cash and equity

Top-tier relocation support

Job summary

AGI Inc, located in San Francisco, is seeking a dedicated professional for a full-time role to evaluate agent models and develop practical assessment rubrics. This position offers competitive cash and equitable stock options along with top-tier relocation support.

After 30 days, you will audit evaluations and pinpoint improvements; by 60 days, you will implement new evaluation tools to aid decision-making. This role is pivotal to ensure product quality and enhance the research strategy.

Qualifications

• Experience in auditing evaluation metrics and producing documentation.
• Ability to implement new evaluation surfaces and support researchers.

Responsibilities

• Audit every evaluation metric and identify gaps after 30 days.
• Establish a new eval surface for decision-making after 60 days.
• Ensure releases meet evaluation standards and influence research roadmap after 90 days.

Skills

Measuring non-deterministic systems

Pushing back on gamed metrics

Job description

AGI Inc, located in San Francisco, is seeking a dedicated professional for a full-time role to evaluate agent models and develop practical assessment rubrics. This position offers competitive cash and equitable stock options along with top-tier relocation support.

After 30 days, you will audit evaluations and pinpoint improvements; by 60 days, you will implement new evaluation tools to aid decision-making. This role is pivotal to ensure product quality and enhance the research strategy.