Technical Project Manager

lilt-corporate

United States

Remote

USD 140,000 - 210,000

Full time

14 days+

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Job summary

lilt-corporate seeks a role-focused professional to manage technically complex Applied AI programs that involve multilingual data collection, human-in-the-loop operations, and large language model evaluation. You will translate AI requirements into executable workflows, guide data-specs, and coordinate engineering and contributor teams to deliver high-quality datasets and evaluation results.

The role emphasizes partnering with engineers on data pipelines, annotation tooling, and automated

Qualifications

  • Experience managing complex technical projects involving data operations, AI, ML workflows.
  • Strong understanding of LLM training and evaluation concepts including supervised fine-tuning, RLHF, human evaluation, automated evaluation, and red teaming.
  • Proficiency in SQL for analyzing delivery velocity, quality metrics, and cost structures.
  • Working knowledge of data pipelines, structured data formats, APIs, validation processes, and engineering dependencies.
  • Ability to manage complex workflows using Agile, Scrum, or Kanban practices and communicate technical requirements to global teams.
  • Experience optimizing delivery economics, such as cost per task or token, while maintaining quality standards.

Responsibilities

  • Lead data-collection and evaluation initiatives from technical scoping through implementation, validation, delivery, and retrospective review.
  • Convert AI requirements into data specifications, annotation guidance, evaluation plans, acceptance criteria, schemas, metadata requirements, and integration needs.
  • Coordinate supervised fine-tuning, preference-data, model-evaluation, safety-testing, and response-ranking workflows with technical specialists and domain experts.
  • Partner with engineers on ingestion, annotation tooling, data-generation pipelines, automated checks, system integrations, and issue resolution.
  • Operationalize hybrid evaluation approaches that combine human review, automated metrics, and LLM-as-a-judge methods.
  • Monitor throughput, quality, completeness, inter-annotator agreement, benchmark discrepancies, delivery risks, and unit economics; lead root-cause analysis when performance falls short.

Skills

Project management
AI/ML workflows
Data operations
SQL
Agile/Scrum/Kanban
Global collaboration

Education

Background in CS/DS/Engineering

Tools

Jira
APIs
SQL tooling

Job description

Role overview

Manage technically complex Applied AI programs involving multilingual data collection, human-in-the-loop operations, and large language model evaluation. The role translates AI requirements into executable workflows, coordinates engineering and contributor teams, and uses operational metrics to deliver reliable datasets and evaluation results at the required quality, speed, and cost.

Responsibilities
  • Lead data-collection and evaluation initiatives from technical scoping through implementation, validation, delivery, and retrospective review.
  • Convert AI requirements into data specifications, annotation guidance, evaluation plans, acceptance criteria, schemas, metadata requirements, and integration needs.
  • Coordinate supervised fine-tuning, preference-data, model-evaluation, safety-testing, and response-ranking workflows with technical specialists and domain experts.
  • Partner with engineers on ingestion, annotation tooling, data-generation pipelines, automated checks, system integrations, and issue resolution.
  • Operationalize hybrid evaluation approaches that combine human review, automated metrics, and LLM-as-a-judge methods.
  • Monitor throughput, quality, completeness, inter-annotator agreement, benchmark discrepancies, delivery risks, and unit economics; lead root-cause analysis when performance falls short.
Requirements
  • Experience managing complex technical projects involving data operations, AI, machine learning, language technology, or related workflows.
  • Strong understanding of LLM training and evaluation concepts, including supervised fine-tuning, reinforcement learning from human feedback, human evaluation, automated evaluation, LLM-as-a-judge, and red teaming.
  • Proficiency in SQL for analyzing delivery velocity, quality metrics, and cost structures.
  • Working knowledge of data pipelines, structured data formats, APIs, validation processes, and engineering dependencies.
  • Ability to manage complex workflows using Agile, Scrum, or Kanban practices and communicate technical requirements to global teams.
  • Experience optimizing delivery economics, such as cost per task or token, while maintaining quality standards.
Nice to have
  • Python or scripting experience for data analysis, automation, or evaluation tooling.
  • Experience managing specialized subject-matter experts or distributed contributor networks.
  • Familiarity with annotation platforms, business-intelligence tools, automated evaluation systems, and Jira.
  • Experience delivering multilingual or multimodal data programs, plus fluency in an additional language.
  • Background in computer science, data science, engineering, or equivalent practical experience.
Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Project Manager, Applied AI
Project Manager, Applied AI

lilt-corporate • United States

Remote
USD 120,000 - 165,000
Applied AI Program Manager - Data Ops & Evaluation
Applied AI Program Manager - Data Ops & Evaluation

lilt-corporate • United States

Remote
USD 140,000 - 210,000
Applied AI Project Lead: Multilingual Data & LLM Evaluation
Applied AI Project Lead: Multilingual Data & LLM Evaluation

LILT AI • United States

On-site
USD 110,000 - 180,000
Applied AI Project Manager: Data Pipelines & LLM Eval
Applied AI Project Manager: Data Pipelines & LLM Eval

Lilt---Lega-Italiana-Per-La-Lotta-Contro-I-Tumori-1 • United States

On-site
USD 140,000 - 210,000
Project Manager, Applied AI
Project Manager, Applied AI

LILT (Production) • United States

Remote
USD 23,000 - 32,000
Applied AI PM: Data Workflows & LLM Evaluation
Applied AI PM: Data Workflows & LLM Evaluation

LILT • United States

On-site
USD 120,000 - 180,000
Global AI Data Operations Project Manager
Global AI Data Operations Project Manager

Lilt, Inc. • United States

Remote
USD 22,730 - 32,373
Applied Scientist/Research Engineer, LLM Training Data
Applied Scientist/Research Engineer, LLM Training Data

Propio • Overland Park (KS)

On-site
USD 120,000 - 180,000
Head of Technical Program Management, Applied AI
Head of Technical Program Management, Applied AI

lilt-corporate • United States

Remote
USD 180,000 - 260,000
Equity participation
Professional growth opportunities
Global AI Data Projects Manager
Global AI Data Projects Manager

LILT (Production) • United States

Remote
USD 23,000 - 32,000