Director of AI Evaluation & Research

Vals AI, Inc.; San Francisco (CA); On-site; USD 210,000 - 320,000; Full time

Benefits offered by this job

Relocation support

Health insurance

Dental insurance

Lunch & snacks

401K plan

Unlimited PTO

Housing stipend

Equity

Job summary

Vals AI is seeking a Head of Research in San Francisco to lead the measurement layer for AI economics. You will set evaluation strategies for long-horizon tasks, oversee the research portfolio, and guide a growing team while collaborating with enterprise customers and lab partners.

You will publish influential work, shape methodologies, and ensure research translates into real deployments, with a focus on impactful, deployable results.

Qualifications

• PhD in ML/NLP or equivalent (in progress or completed).
• Deep familiarity with LLM evaluation landscape and benchmarks.
• Bias toward research with deployable impact, not gameable benchmarks.
• Strong written and verbal communication; ability to present to customers and labs.
• Ability to work in-person in San Francisco.

Responsibilities

• Advance the science of evaluation and develop new paradigms for long-horizon tasks.
• Oversee Vals' broader research portfolio and set direction for ongoing and future projects.
• Publish work that advances the field and gains citations.
• Recruit and grow a research team alongside the founders.
• Collaborate with enterprise customers and lab partners on evaluation problems.

Skills

ML/NLP PhD

LLM evaluation

Communication skills

In-person SF work

Education

PhD in ML/NLP or equivalent

Job description

Vals AI is seeking a Head of Research in San Francisco to lead the measurement layer for AI economics. You will set evaluation strategies for long-horizon tasks, oversee the research portfolio, and guide a growing team while collaborating with enterprise customers and lab partners.

You will publish influential work, shape methodologies, and ensure research translates into real deployments, with a focus on impactful, deployable results.