Engineering Manager, AI Evaluation - handshake

Polluxa, Inc.; San Francisco (CA); On-site; USD 180,000 - 260,000; Full time

Job summary

Handshake is seeking an Engineering Manager to own our Quality team in San Francisco. You will lead quality engineering systems that improve AI output reliability, evaluation accuracy, and benchmark performance for frontier lab partners.

You will build and scale an organization across technical domains and partner with stakeholders to prioritize platform investments. The role is in-office, 5 days a week in San Francisco, CA, with a focus on developing senior ICs and driving cross-team technical

Qualifications

• 7+ years of engineering experience with 3+ years in management of sizeable teams.
• Experience shipping in AI-centric environments with data pipelines and evaluation rigor.
• Strong product engineering instincts for user-facing systems, not only backend infra.
• Ability to lead multiple technical domains and grow senior ICs and EMs.
• Proven prioritization and stakeholder management with clear communication.

Responsibilities

• Lead quality engineering systems that improve AI output reliability and benchmark performance.
• Build and scale an organization across distinct technical domains and drive strategy.
• Partner with stakeholders to prioritize platform investments that accelerate teams.

Skills

Engineering leadership

People management

Systems design

Cross-functional alignment

Communication

Job description

Handshake was founded on a simple belief that everyone deserves a path to a great career, regardless of where they went to school or who they know. Today, we power 25 million job seekers, 1 million+ employers, and 1,600 educational institutions.

In 2025, we started Handshake AI and built the fastest-growing AI data business in history. We work directly with frontier AI lab researchers to create evaluations, publish benchmarks, and push the boundary of data. We’ve grown from $0 to ~$1B run rate and pay ~$60M to over 30K individuals every month.

ABOUT THE ROLE

We're hiring an Engineering Manager to own our Quality team. This team builds the extensible platform infrastructure that operators and fellows rely on every day.

• Lead quality engineering systems that directly improve AI output reliability, evaluation accuracy, and benchmark performance for frontier lab partners
• Build and scale an organization across distinct technical domains, developing senior ICs and driving cross-team technical strategy
• Partner with internal stakeholders to prioritize platform investments that unlock other teams to move faster and build on top of shared infrastructure

Location: San Francisco, CA. This is an in-office role, 5 days/week (no remote/hybrid)

YOU HAVE

Must-Haves:

• 7+ years of engineering experience, including 3+ years managing engineering teams of 8 or more and experience manager managers.
• Background at companies where AI is the core product; you’ve shipped in environments where model quality, data pipelines, and evaluation rigor are table stakes
• Strong product engineering instincts; you can lead teams building complex user-facing systems, not just backend infrastructure
• Experience leading teams with meaningfully different technical focuses simultaneously
• Track record of developing engineers; you’ve grown senior ICs, EMs, and leads, not just delivered projects
• Strong prioritization and stakeholder management
• Clear, direct communicator who can translate between technical complexity and business context

What we’re looking for

• Engineering leader + builder: 3+ years managing teams, plus 5+ years hands-on engineering experience
• Strong people leadership: experience leading senior engineers; managing an EM (or equivalent scope) is a plus
• Execution in ambiguity: proven ability to align cross-functionally and deliver in fast-moving, unclear problem spaces
• Systems + product mindset: strong platform/distributed systems background, and the ability to turn needs into a clear roadmap, ship iteratively, and measure outcomes

Bonus Points

• Familiarity with LLM evaluation, quality pipelines, or applied AI tooling
• Experience with annotation or data labeling platforms, tasking systems, or workflow orchestration