Research Engineer — AGI Evaluation & On-Device Perf

AGI, Inc.; San Francisco (CA); On-site; USD 120,000 - 160,000; Full time

Benefits offered by this job

Competitive cash and equity

Top-tier relocation support

In-person work environment

Job summary

AGI, Inc. in San Francisco is looking for a skilled professional to build evaluation harnesses that ensure models and agents are performing at their best. You will set criteria for releases, audit existing evaluation processes, and develop tooling to assist research and product teams.

The position emphasizes collaboration and delivery of dependable performance metrics to improve AI capabilities. You'll need to have a firm grasp on non-deterministic systems and QA processes in a production environment, enhancing user experience and device performance.

Competitive compensation and relocation support are offered.

Qualifications

• Ability to audit evals and identify gaps.
• Experience with agent performance evaluation.
• Skilled in measuring long-horizon tasks.

Responsibilities

• Build eval harness for model capability and user experience.
• Create dashboards for researcher experiment loops.
• Define readiness criteria for product launches.

Skills

Agent evaluation

Tool use measurement

QA at OEM scale

Multilingual behavior assessment

Job description

AGI, Inc. in San Francisco is looking for a skilled professional to build evaluation harnesses that ensure models and agents are performing at their best. You will set criteria for releases, audit existing evaluation processes, and develop tooling to assist research and product teams.

The position emphasizes collaboration and delivery of dependable performance metrics to improve AI capabilities. You'll need to have a firm grasp on non-deterministic systems and QA processes in a production environment, enhancing user experience and device performance.

Competitive compensation and relocation support are offered.