Head of Research

Vals AI, Inc.; San Francisco (CA); On-site; USD 210,000 - 320,000; Full time

Benefits offered by this job

Relocation support

Health insurance

Dental insurance

Lunch & snacks

401K plan

Unlimited PTO

Housing stipend

Equity

Job summary

Vals AI is seeking a Head of Research in San Francisco to lead the measurement layer for AI economics. You will set evaluation strategies for long-horizon tasks, oversee the research portfolio, and guide a growing team while collaborating with enterprise customers and lab partners.

You will publish influential work, shape methodologies, and ensure research translates into real deployments, with a focus on impactful, deployable results.

Qualifications

• PhD in ML/NLP or equivalent (in progress or completed).
• Deep familiarity with LLM evaluation landscape and benchmarks.
• Bias toward research with deployable impact, not gameable benchmarks.
• Strong written and verbal communication; ability to present to customers and labs.
• Ability to work in-person in San Francisco.

Responsibilities

• Advance the science of evaluation and develop new paradigms for long-horizon tasks.
• Oversee Vals' broader research portfolio and set direction for ongoing and future projects.
• Publish work that advances the field and gains citations.
• Recruit and grow a research team alongside the founders.
• Collaborate with enterprise customers and lab partners on evaluation problems.

Skills

ML/NLP PhD

LLM evaluation

Communication skills

In-person SF work

Education

PhD in ML/NLP or equivalent

Job description

About the Role

Measuring intelligence is hard, and humans haven't been particularly good at it. The proxies we've used — IQ, standardized tests, credentials — have shaped how we develop intelligence and how we value it, often in ways we later regret. AI gives us a chance to do better. The field is young enough that the methodologies for measuring what these systems can actually do are still being written, and the answers we settle on will shape what gets built, what gets deployed, and which workflows get automated next.

Vals is building the measurement layer for the AI economy: the benchmarks, methodologies, and standards that determine which models ship and where they get trusted. We're hiring a Head of Research to lead it.

The hard research questions don't have textbook answers yet. How do you measure whether an LLM can actually do a real lawyer's contract review, a real underwriter's risk assessment, a real radiologist's read? How do you build evaluations that hold up as models get better at gaming them? You'll be the person setting the direction on how Vals — and by extension, much of the field — answers them.

What You'll Do

• Advance the science of evaluation. The methodologies the field uses today — judge models, human-in-the-loop, static benchmarks — were built for a previous generation of models and break down on long-horizon, real-world tasks. You'll develop the new paradigms.

• Oversee Vals' broader research portfolio, setting direction across the projects already underway and the ones we haven't started yet.

• Publish work that moves the field forward. We want Vals' research to be cited, not just shipped.

• Recruit and grow a research team alongside the founders.

• Work directly with our enterprise customers and lab partners on the evaluation problems they actually have.

Requirements

• A PhD in ML/NLP (in progress or completed), or equivalent industry research track record

• Deep familiarity with the LLM evaluation landscape: existing benchmarks, their failure modes, judge-model approaches, human-in-the-loop methodologies.

• A bias toward research that affects what people actually deploy, rather than benchmarks that are easy to game.

• Strong written and verbal communication. You'll publish, present, and talk to customers and labs.

• Ability to work in-person, in San Francisco.

Nice-to-Haves

• A widely-cited benchmark or eval framework you've built or co-built.

• Prior experience at a frontier lab (Anthropic, OpenAI, Google DeepMind, Meta FAIR) or a research-led startup.

• Domain depth in one or more of our verticals (legal, finance, insurance, healthcare).

• Experience leading or mentoring other researchers.

• A public research presence: papers, blog posts, talks, or open-source contributions others in the field recognize.

What We Offer

• Highly competitive salary and equity. Excellence is well rewarded.

• Relocation and transportation support

• Collaboration with leading AI labs and the opportunity to work on the front lines of AI development and research

• Health/dental insurance coverage

• Lunch and dinner provided, free snacks/coffee/drinks

• 401K plan

• Unlimited PTO

• $1,500 housing stipend (within one-mile radius)

About Us

Founding team: The core methodology behind this platform comes from NLP evaluation research we conducted at Stanford. We raised a $5M seed from some of the top institutional and angel investors in the valley. Our team has prior work experience at NVIDIA, Meta, Microsoft, Palantir and HRT. Collectively, we have over 300 citations in our published work. Our early team includes Stanford PhDs, ex-Jane Street quants, and the first designer at Snorkel.

We recently announced our $40M Series A at a $400M valuation, led by Andreessen Horowitz, with participation from existing investors 8VC, Pear VC, and Bloomberg and new investors Hudson River Trading and NextLadder Ventures.

Tech stack: We use Python for most things at Vals AI. Our platform is built on Django, with a React frontend. All of the infra is on AWS using CDK for IaC.

What We're Looking For

• Learning velocity: The role encompasses a wide variety of tasks. Rather than expecting you to be an expert on Day 1, we are looking for someone who can learn new skills and technologies quickly.

• Ownership: Working in a small, talent-dense team, we expect everyone to show initiative to build where it's needed, not where it's asked. We strive for autonomy over consensus.

• Intensity: The LLM landscape is constantly changing. Foundation model labs are continuously pushing the frontier. The unicorn companies that will emerge from this technology shift are being built now. Those that win will have an incredibly high speed of execution.

• Solution-oriented mindset: We're looking for people who see opportunities to craft solutions at each juncture, not those who pass hard problems to others or admit defeat.

Vals in the Media:

• AI tools mostly fumble basic financial tasks, study finds

• The Winners (and Losers) of This New Vibe-Coding Benchmark Will Surprise You

• OpenAI's Less-Flashy Rival Might Have a Better Business Model

• Meta Announces New AI Model in Major Test of Company's Ambitions

• DeepSeek's Sequel Set to Extend China's Reach in Open-Source A.I.