Quality Manager, Applied AI

lilt-corporate

United States

On-site

USD 130,000 - 180,000

Full time

14 days+

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Job summary

LILT is hiring for a quality and evaluation-focused role shaping enterprise-grade AI translation and evaluation workflows. You will lead quality frameworks across multiple programs, build tooling ecosystems, and ensure scalable, measurable QA that supports fast, reliable AI outputs.

The ideal candidate has 6+ years in applied AI/evaluation, experience with human-in-the-loop pipelines, and strong data governance. Join a growth-focused, globally collaborative team.

Qualifications

  • Applied AI/evaluation experience in ML/LLM systems.
  • Design and run end-to-end quality frameworks.
  • Strong knowledge of reliability and agreement methods.
  • Experience with calibrations, adjudication, and drift monitoring.
  • Cross-functional collaboration with AI, product and partners.

Responsibilities

  • Ensure workflows are built for quality with measurable targets.
  • Own technical tooling and quality infrastructure.
  • Own operational quality at scale across programs.
  • Delivery sign-off with go/no-go decisions and readiness memos.
  • Remediate customer quality issues with data-driven analysis.

Skills

Applied AI
Quality programs
ML/LLM evaluation
Data labeling
Annotation tooling

Education

B.S./M.S. in quantitative field

Tools

Label Studio

Job description

About LILT

AI is changing how the world communicates - and LILT is leading that transformation.

We're on a mission to make the world's information accessible to everyone , regardless of the language they speak. We use cutting-edge AI, machine translation, and human-in-the-loop expertise to translate content faster, more accurately, and more cost-effectively without compromising on brand, voice, or quality.

At LILT, we empower our teammates with leading tools, global collaboration, and growth opportunities to do their best work. Our company virtues- Work together, win together; Find a way or make one; Dance in the customer's shoes; Quicker than they expect; Quality is Job 1 -guide everything we do. We are trusted by Intel Corporation , Canva , the United States Department of Defense , the United States Air Force , ASICS , and hundreds of global Enterprises. Backed by Sequoia, Intel Capital, and Redpoint, we're building a category-defining company in a $50B+ global translation market being redefined by AI.

Key Responsibilities
  • Ensure Workflows Are Built for Quality

    Outcome: Every Applied AI deliverable ships with measured quality: rater agreement meets agreed thresholds per dimension, rubric scores are reproducible across raters, and reported quality figures hold up under customer audit.

  • Own Technical Tooling & Quality Infrastructure

    Outcome: Quality metrics for every active program (defect rate, acceptance rate, rework, reviewer reliability, throughput, SLA adherence) are available without manual compilation, and systemic quality issues are detected from workflow data before a customer reports them.

  • Own Operational Quality at Scale

    Outcome: Concurrent programs meet their quality targets within agreed cost and throughput. Acceptance rates hold at or above target, rework declines over time, and standard QC on standardized workflows is executed and interpreted by Production staff without Quality Manager involvement by the end of the second quarter in role.

  • Delivery sign-off

    Every deliverable has a documented go/no-go decision before shipping, based on programmatic QA and a readiness memo with flags. Post-sign-off quality escapes stay below an agreed threshold, and each escape results in a documented process change.

  • Own Customer quality Remediation

    Outcome: Customer quality disputes are closed with quantitative analysis, normally at first response. Re-adjudication completes within SLA, recurring defect classes decline release over release, and annotator integrity issues are detected internal before they reach a deliverable.

Qualifications
  • Education: B.S./M.S. in a quantitative field (CS, Statistics, Data Science, Engineering, Linguistics, HCI) or equivalent practical experience.
  • Applied AI / evaluation experience: 6+ years building or operating quality programs for ML/LLM systems (data labeling, benchmark creation, model evaluation, or human-in-the-loop pipelines).
  • Quality systems leadership: Proven ability to design and run end-to-end quality frameworks (rubrics, sampling plans, QC gates, acceptance criteria, escalation paths) across multiple concurrent programs.
  • Measurement & statistics: Strong grasp of reliability and agreement methods (e.g., Krippendorff's α, Cohen's κ), power/sample-size intuition, error analysis, and reporting for executive stakeholders.
  • Rater/annotator operations: Experience with calibration, adjudication, targeted retraining, drift monitoring, and integrity/fraud detection in high-throughput human evaluation programs.
  • Tooling & automation: Hands‑on with labeling/eval tooling (e.g., Label Studio or similar) and designing deterministic validations; comfort partnering with engineering/research to implement quality instrumentation.
  • Data governance: Strong understanding of dataset/version management, traceability, privacy/compliance constraints, and audit‑ready documentation for deliverables.
  • Stakeholder management: Excellent written and verbal communication; able to translate ambiguous research goals into measurable quality targets and negotiate tradeoffs with Applied AI, product, and external partners.
Preferred Skills
  • Fluency in multiple human languages.
  • Proficiency in a programming language and experience working with modern AI frameworks.
Our Story

Our founders, Spence and John met at Google working on Google Translate. As researchers at Stanford and Berkeley, they both worked on language technology to make information accessible to everyone. While together at Google, they were amazed to learn that Google Translate wasn't used for enterprise products and services inside the company.The quality just wasn't there. So they set out to build something better. LILT was born.

LILT has been a machine learning company since its founding in 2015. At the time, machine translation didn't meet the quality standard for enterprise translations, so LILT assembled a cutting-edge research team tasked with closing that gap. While meeting customer demand for translation services, LILT ha

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Senior Quality Manager, Applied AI
Senior Quality Manager, Applied AI

LILT AI • City of Syracuse (NY)

Hybrid
USD 150,000 - 190,000
Director of Quality
Director of Quality

LILT AI • City of Syracuse (NY)

Hybrid
USD 180,000 - 250,000
Technical Project Manager
Technical Project Manager

LILT • United States

On-site
USD 120,000 - 180,000
Technical Project Manager
Technical Project Manager

Lilt- Lega Italiana per la Lotta contro i Tumori • United States

On-site
USD 140,000 - 210,000
Director of Quality
Director of Quality

LILT • Boston (MA)

Hybrid
USD 180,000 - 240,000
Director of Quality
Director of Quality

Lilt, Inc. • Boston (MA)

Hybrid
USD 180,000 - 240,000
Technical Project Manager
Technical Project Manager

LILT AI • United States

On-site
USD 110,000 - 180,000
Global AI Data Projects Manager
Global AI Data Projects Manager

Lilt • United States

Hybrid
USD 90,000 - 150,000
Head of Technical Program Management, Applied AI
Head of Technical Program Management, Applied AI

LILT AI • San Francisco (CA)

On-site
USD 260,000 - 420,000
Competitive salary
Equity
Professional growth
Project Manager, Applied AI
Project Manager, Applied AI

LILT AI • United States

On-site
USD 120,000 - 180,000