Applied AI Program Manager - Data Ops & Evaluation

lilt-corporate

United States

Remote

USD 140,000 - 210,000

Full time

14 days+

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Job summary

lilt-corporate seeks a role-focused professional to manage technically complex Applied AI programs that involve multilingual data collection, human-in-the-loop operations, and large language model evaluation. You will translate AI requirements into executable workflows, guide data-specs, and coordinate engineering and contributor teams to deliver high-quality datasets and evaluation results.

The role emphasizes partnering with engineers on data pipelines, annotation tooling, and automated

Qualifications

  • Experience managing complex technical projects involving data operations, AI, ML workflows.
  • Strong understanding of LLM training and evaluation concepts including supervised fine-tuning, RLHF, human evaluation, automated evaluation, and red teaming.
  • Proficiency in SQL for analyzing delivery velocity, quality metrics, and cost structures.
  • Working knowledge of data pipelines, structured data formats, APIs, validation processes, and engineering dependencies.
  • Ability to manage complex workflows using Agile, Scrum, or Kanban practices and communicate technical requirements to global teams.
  • Experience optimizing delivery economics, such as cost per task or token, while maintaining quality standards.

Responsibilities

  • Lead data-collection and evaluation initiatives from technical scoping through implementation, validation, delivery, and retrospective review.
  • Convert AI requirements into data specifications, annotation guidance, evaluation plans, acceptance criteria, schemas, metadata requirements, and integration needs.
  • Coordinate supervised fine-tuning, preference-data, model-evaluation, safety-testing, and response-ranking workflows with technical specialists and domain experts.
  • Partner with engineers on ingestion, annotation tooling, data-generation pipelines, automated checks, system integrations, and issue resolution.
  • Operationalize hybrid evaluation approaches that combine human review, automated metrics, and LLM-as-a-judge methods.
  • Monitor throughput, quality, completeness, inter-annotator agreement, benchmark discrepancies, delivery risks, and unit economics; lead root-cause analysis when performance falls short.

Skills

Project management
AI/ML workflows
Data operations
SQL
Agile/Scrum/Kanban
Global collaboration

Education

Background in CS/DS/Engineering

Tools

Jira
APIs
SQL tooling

Job description

lilt-corporate seeks a role-focused professional to manage technically complex Applied AI programs that involve multilingual data collection, human-in-the-loop operations, and large language model evaluation. You will translate AI requirements into executable workflows, guide data-specs, and coordinate engineering and contributor teams to deliver high-quality datasets and evaluation results.

The role emphasizes partnering with engineers on data pipelines, annotation tooling, and automated

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Applied AI Project Lead: Multilingual Data & LLM Evaluation
Applied AI Project Lead: Multilingual Data & LLM Evaluation

LILT AI • United States

On-site
USD 110,000 - 180,000
Applied AI Project Manager: Data Pipelines & LLM Eval
Applied AI Project Manager: Data Pipelines & LLM Eval

Lilt---Lega-Italiana-Per-La-Lotta-Contro-I-Tumori-1 • United States

On-site
USD 140,000 - 210,000
Technical Project Manager
Technical Project Manager

lilt-corporate • United States

Remote
USD 140,000 - 210,000
Global AI Data Projects Manager
Global AI Data Projects Manager

LILT (Production) • United States

Remote
USD 23,000 - 32,000
Global AI Data Projects Lead
Global AI Data Projects Lead

LILT AI • United States

On-site
USD 120,000 - 180,000
Global Multilingual AI Data Programs Lead
Global Multilingual AI Data Programs Lead

lilt-corporate • United States

Remote
USD 120,000 - 165,000
Director of AI Program Delivery & Growth
Director of AI Program Delivery & Growth

lilt-corporate • United States

Remote
USD 180,000 - 260,000
Equity participation
Professional growth opportunities
Global AI Data Projects Manager
Global AI Data Projects Manager

Lilt---Lega-Italiana-Per-La-Lotta-Contro-I-Tumori-1 • San Francisco (CA)

On-site
USD 120,000 - 180,000
AI Data Operations Program Manager
AI Data Operations Program Manager

Meta • United States

On-site
USD 132,000 - 189,000
AI Data & Evaluation Program Manager
AI Data & Evaluation Program Manager

CLERA • San Francisco (CA)

On-site
USD 130,000 - 225,000
Visa sponsorship