Applied AI Project Lead: Multilingual Data & LLM Evaluation

LILT AI; United States; On-site; USD 110,000 - 180,000; Full time

Job summary

LILT is seeking a Technical Project Manager for its Applied AI team in the United States. You will lead large‑scale multilingual data collection, human‑in‑the‑loop workflows, and LLM evaluation initiatives, collaborating with AI specialists, engineers, SMEs, and data operations teams.

The role combines technical program management with hands‑on data analysis, unit economics optimization, and managing modern LLM training/evaluation processes.

Qualifications

• 3–5+ years of technical project management experience in AI/ML, data platforms, or technical data operations.
• Strong understanding of LLM training and evaluation concepts including SFT, RLHF, human evaluation, automated evaluations, and red teaming.
• Proficiency in SQL for analyzing delivery velocity, data quality metrics, and cost structures.
• Working knowledge of data pipelines, structured data formats, APIs, and validation processes.
• Experience partnering with engineers to manage technical requirements, dependencies, and issue resolution.
• Proven ability to track and optimize project unit economics (cost per token/task) and delivery velocity.
• Strong communication skills for multilingual audiences and diverse contributor networks.
• Experience managing complex workflows using Agile, Scrum, or Kanban.

Responsibilities

• Manage Applied AI data collection and evaluation projects from technical scoping through validation, delivery, and retrospective review.
• Translate technical AI requirements into structured data specifications, annotation requirements, evaluation plans, and acceptance criteria.
• Manage timelines and throughput velocity, dependencies, delivery risks, and blockers across Applied AI, engineering, and operations stakeholders.
• Coordinate data collection and annotation for supervised fine‑tuning (SFT), RLHF data, and model evaluation.
• Partner with engineering to define data workflows, ingestion, tooling, systems integrations, and quality checks.
• Operationalize hybrid evaluation methodologies, combining human review with automated evaluation tooling and LLM‑as‑a‑judge frameworks.
• Coordinate data formats, schemas, metadata, annotation tooling, and system integrations.
• Work with specialists to operationalize evaluation methods, including human review, response ranking, and safety testing.

Skills

AI/ML PM

SQL

Data workflows

Agile/Scrum

Stakeholder comms

Education

Background in CS or related field

Tools

Jira

Python

BI tools

APIs

Job description

LILT is seeking a Technical Project Manager for its Applied AI team in the United States. You will lead large‑scale multilingual data collection, human‑in‑the‑loop workflows, and LLM evaluation initiatives, collaborating with AI specialists, engineers, SMEs, and data operations teams.

The role combines technical program management with hands‑on data analysis, unit economics optimization, and managing modern LLM training/evaluation processes.