Quality Software Engineer

Handshake; San Francisco (CA); On-site; USD 140,000 - 190,000; Full time

Benefits offered by this job

Health Insurance

401k match

Equity

Learning & development

Parental leave

Flexible time off

Job summary

Handshake is hiring a Software Engineer to join the Quality team in San Francisco. You will build and scale backend systems that evaluate and improve the quality of data delivered to leading AI labs, with real-time checks and robust pipelines that surface measurable quality signals.

You will collaborate with engineers, researchers, product teams, and operations to turn ambiguous quality problems into scalable technical solutions, while improving reliability and reducing costs as the business

Qualifications

• Experience building backend services and data-driven systems.
• Strong fundamentals in systems design, testing, and reliability.
• Ability to work across teams to turn ambiguous problems into scalable solutions.

Responsibilities

• Build and scale backend systems that evaluate data quality.
• Develop infrastructure for real-time quality checks and end-to-end validation.
• Design data pipelines and measurement systems for observable quality signals.
• Create APIs and platform capabilities for internal tools and teams.
• Collaborate with engineers, researchers, product, and operations stakeholders.
• Improve reliability, efficiency, and cost-effectiveness of quality workflows.

Skills

Backend systems

APIs & databases

Distributed systems

Data pipelines

System design

Testing & debugging

Operational reliability

Collaboration

ML/AI familiarity

Job description

• We’re hiring a Software Engineer to join our Quality team. The team builds the systems that define, select, and enforce quality standards across the data and evaluations we deliver to leading AI labs
• You’ll work on infrastructure that evaluates complex outputs submitted by fellows, such as coding tasks designed to challenge advanced AI models
• These quality checks run throughout the production process: providing real-time feedback to fellows, supporting internal operators, and validating outputs before they reach customers
• Build and scale backend systems that evaluate and improve the quality of data delivered to frontier AI labs
• Develop infrastructure for running quality checks in real time and before final customer delivery
• Help design recommendation systems that select the most effective quality checks for each project
• Build data pipelines and measurement systems that make complex, subjective quality signals more observable and actionable
• Develop APIs and platform capabilities that allow other teams and tools to use the Quality platform
• Help surface Quality capabilities through MCP and other AI-native interfaces used by internal operators
• Partner with engineers, researchers, product teams, and operations stakeholders to turn ambiguous quality problems into scalable technical solutions
• Improve the reliability, efficiency, and cost-effectiveness of quality workflows as the business grows

Benefits

• Health Insurance: Comprehensive medical , dental, and vision coverage
• Fertility and family forming: Through a partnership, Handshake offers comprehensive family benefits, including infertility treatments and a $15K stipend, to its employees.
• LGTBQ+ coverage: All medical plans cover transgender services.
• Mental health: Discounts on a variety of wellness apps and an Employee Assistance Program with access to five free counseling sessions.
• Tax savings: Tax-advantaged benefits to save money on healthcare, dependent care, and other expenses.
• Life & disability insurance: Essential benefits and automatic coverage with basic life insurance, plus short-term and long-term disability coverage.
• 401k match: Dollar-for-dollar match up to 1% of pay, with a max of $1,200 per year.
• Equity: Our equity program includes grants at hire, refresher grants after two years of tenure, and grants alongside promotions.
• Financial management: Handshake partners to provide free professional financial planning services.
• Learning & development: Employees are provided with an annual stipend to use towards professional development, career development workshops, manager trainings, and 1-on-1 coaching through external partnerships
• Community leader support: Employee Resource Group leaders are compensated with a quarterly bonus for the high-impact work the role requires
• Parental leave: Leaves of 16 weeks for birth-giving parents and 10 weeks for non-birth-giving parents; the policy applies to adoptions as well. Handshake also partners to provide one-on-one coaching on preparing for leave and coming back from leave
• Flexible time off: No accrued time off or waiting period to take vacation. Two paid flexible holiday days off to observe holidays that closely align with religious, cultural, and personal needs
• Sabbatical: Paid six-week sabbatical to recharge and reset beginning at six years of continuous, full-time employment, and every four years thereafter
• Volunteer time off: Two days off for social and civil activism, volunteer work, and/or any related activities, including attending and participating in protests

Experience

• Experience working with APIs, databases, data pipelines, or distributed systems
• Strong software engineering fundamentals, including system design, testing, debugging, and operational reliability
• The ability to navigate ambiguous problems, break them into manageable pieces, and deliver iteratively
• 2+ years of experience building and maintaining backend applications, services, or data-intensive systems
• A quality-focused mindset and an interest in building systems that make difficult-to-measure outcomes more rigorous
• Strong communication and collaboration skills across technical and non-technical teams
• A data-oriented approach to solving problems and measuring whether a system is working
• Familiarity with model evaluation, experimentation, recommendation systems, or ranking systems
• Experience with machine learning, applied AI, or AI-enabled products
• Experience building developer platforms, internal tools, or extensible APIs
• Experience working in a fast-moving startup or similarly ambiguous environment
• Experience with annotation, data-labeling, tasking, or workflow-orchestration platforms
• Familiarity with MCP, agentic systems, or AI-native development tools