Staff AI Evaluations Engineer — Open Foundation Models

B Capital; San Francisco (CA); On-site; USD 120,000 - 160,000; Full time

Benefits offered by this job

Top-tier compensation

Comprehensive medical, dental, vision, life, and disability insurance

Fully paid parental leave

Paid time off

Lunch and dinner provided daily

Relocation support

Job summary

B Capital seeks a talented individual for an AI Evaluation role in San Francisco. This position involves conducting critical comparative analysis, refining evaluation systems, and collaborating with various teams to enhance model capabilities. The ideal candidate will have strong statistical analysis skills, familiarity with LLM evaluation methodologies, and possess a collaborative and detail-oriented approach. B Capital is dedicated to providing top-tier compensation and comprehensive benefits to support their employees.

Qualifications

• Strong statistical analysis and experimental design skills.
• Familiarity with LLM evaluation methodologies.
• Detail-oriented and collaborative mindset.

Responsibilities

• Conduct critical comparative analysis to advance understanding of model capabilities.
• Build and refine evaluation systems that create feedback loops.
• Collaborate closely with teams to translate insights into model improvements.

Skills

Statistical analysis

Experimental design

LLM evaluation methodologies

Collaboration

Detail-oriented

Job description

B Capital seeks a talented individual for an AI Evaluation role in San Francisco. This position involves conducting critical comparative analysis, refining evaluation systems, and collaborating with various teams to enhance model capabilities. The ideal candidate will have strong statistical analysis skills, familiarity with LLM evaluation methodologies, and possess a collaborative and detail-oriented approach. B Capital is dedicated to providing top-tier compensation and comprehensive benefits to support their employees.