AI Quality Manager for ML & LLM Pipelines

lilt-corporate

United States

On-site

USD 130,000 - 180,000

Full time

14 days+

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Job summary

LILT is hiring for a quality and evaluation-focused role shaping enterprise-grade AI translation and evaluation workflows. You will lead quality frameworks across multiple programs, build tooling ecosystems, and ensure scalable, measurable QA that supports fast, reliable AI outputs.

The ideal candidate has 6+ years in applied AI/evaluation, experience with human-in-the-loop pipelines, and strong data governance. Join a growth-focused, globally collaborative team.

Qualifications

  • Applied AI/evaluation experience in ML/LLM systems.
  • Design and run end-to-end quality frameworks.
  • Strong knowledge of reliability and agreement methods.
  • Experience with calibrations, adjudication, and drift monitoring.
  • Cross-functional collaboration with AI, product and partners.

Responsibilities

  • Ensure workflows are built for quality with measurable targets.
  • Own technical tooling and quality infrastructure.
  • Own operational quality at scale across programs.
  • Delivery sign-off with go/no-go decisions and readiness memos.
  • Remediate customer quality issues with data-driven analysis.

Skills

Applied AI
Quality programs
ML/LLM evaluation
Data labeling
Annotation tooling

Education

B.S./M.S. in quantitative field

Tools

Label Studio

Job description

LILT is hiring for a quality and evaluation-focused role shaping enterprise-grade AI translation and evaluation workflows. You will lead quality frameworks across multiple programs, build tooling ecosystems, and ensure scalable, measurable QA that supports fast, reliable AI outputs.

The ideal candidate has 6+ years in applied AI/evaluation, experience with human-in-the-loop pipelines, and strong data governance. Join a growth-focused, globally collaborative team.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Senior AI Quality Leader
Senior AI Quality Leader

LILT AI • City of Syracuse (NY)

Hybrid
USD 150,000 - 190,000
Head of Quality & AI Excellence
Head of Quality & AI Excellence

Lilt, Inc. • Boston (MA)

Hybrid
USD 180,000 - 240,000
Applied AI Project Manager: Data Pipelines & LLM Eval
Applied AI Project Manager: Data Pipelines & LLM Eval

Lilt---Lega-Italiana-Per-La-Lotta-Contro-I-Tumori-1 • United States

On-site
USD 140,000 - 210,000
Senior Quality Manager, Applied AI
Senior Quality Manager, Applied AI

LILT AI • City of Syracuse (NY)

Hybrid
USD 150,000 - 190,000
Quality Manager, Applied AI
Quality Manager, Applied AI

lilt-corporate • United States

On-site
USD 130,000 - 180,000
AI/ML Quality Engineer — LLM Evaluation & QA Automation
AI/ML Quality Engineer — LLM Evaluation & QA Automation

INSPYR Solutions • Cupertino (CA)

Hybrid
USD 91,000 - 105,000
Remote-friendly schedule
Applied AI Project Lead: Multilingual Data & LLM Evaluation
Applied AI Project Lead: Multilingual Data & LLM Evaluation

LILT AI • United States

On-site
USD 110,000 - 180,000
Applied AI Program Manager - Data Ops & Evaluation
Applied AI Program Manager - Data Ops & Evaluation

lilt-corporate • United States

Remote
USD 140,000 - 210,000
Director of AI Quality & Compliance
Director of AI Quality & Compliance

LILT AI • City of Syracuse (NY)

Hybrid
USD 180,000 - 250,000
Applied AI PM: Data Workflows & LLM Evaluation
Applied AI PM: Data Workflows & LLM Evaluation

LILT • United States

On-site
USD 120,000 - 180,000