Research Engineer: AGI Eval & Agent Quality

Pantera Capital; San Francisco (CA); On-site; USD 120,000 - 160,000; Full time

Benefits offered by this job

Competitive cash compensation

Equity opportunities

Relocation support

Immigration support

Job summary

Pantera Capital is seeking a dedicated professional to develop evaluation metrics for advanced AI models and consumer-grade agents in San Francisco. This role involves ensuring quality and trustworthiness in AI performance and shaping research outcomes by establishing rigorous evaluation standards.

You will advance evaluation frameworks across multiple dimensions and work closely with both researchers and product engineers.

Competitive cash and equity compensation packages are available, along with top-tier relocation support.

Qualifications

• Experience in measuring non-deterministic systems and understanding agent behavior.
• Ability to create evaluation metrics while avoiding gaming.

Responsibilities

• Build evaluation harnesses for various model capabilities.
• Develop dashboards for research and decision-making.
• Set the standards for product readiness.

Skills

Agent evaluation

Tool use

Non-deterministic systems measurement

Multilingual behavior

Job description

Pantera Capital is seeking a dedicated professional to develop evaluation metrics for advanced AI models and consumer-grade agents in San Francisco. This role involves ensuring quality and trustworthiness in AI performance and shaping research outcomes by establishing rigorous evaluation standards.

You will advance evaluation frameworks across multiple dimensions and work closely with both researchers and product engineers.

Competitive cash and equity compensation packages are available, along with top-tier relocation support.