Queue Lead - DP

Prolific

United States

A distancia

USD 69.000 - 96.000

A tiempo parcial

14 días+

Una candidatura completa en un minuto

Currículum y carta de presentación personalizados, listos para enviar.

Supera los filtros ATS

Descripción de la vacante

Prolific is seeking a Queue Lead to manage AI data program queues, quality, and reporting. This remote freelance role focuses on throughput, calibrations, and contributor performance across projects, with emphasis on live program operations and stakeholder visibility.

You will own end-to-end queue execution, monitor quality metrics, and build dashboards while guiding annotators. Ideal candidates have hands-on experience in fast-paced data ops and AI data programs.

Formación

  • Proven, direct experience running queues and quality on AI data programs.
  • 3+ years in high-throughput ops with AI data ops experience.
  • Ability to design calibration processes from scratch.
  • Ability to track dashboards or reporting systems (spreadsheets required).
  • Experience managing contributor performance at scale.

Responsabilidades

  • Own work queues: intake, triage, prioritization, task assignment, and SLA delivery.
  • Monitor task queue and backlog control.
  • Identify and troubleshoot bottlenecks.
  • Monitor quality at project and annotator level.
  • Perform QA tasks and provide annotator feedback.
  • Manage annotator performance including retention/removal decisions.
  • Build and maintain dashboards to track throughput and performance.
  • Turn raw queue data into actionable reporting for stakeholders.
  • Provide feedback for process improvements based on data trends.
  • Oversee Slack channels, train new annotators, and conduct webinars.

Conocimientos

Queue management
Data ops
Quality assessment
Calibration
Dashboards & reporting
Contributor performance
Adaptability

Descripción del empleo

Queue Lead

Job Title: Queue Lead

Start date: TBC
End Date: TBC
Location: Remote
Type of contract: Freelance
Languages required: English
Hourly Rate: Include the rate you are comfortable with on your CV.

About ProlificProlific is not just another player in the AI space. We are building the biggest pool of quality human data in the world. Over 35,000 AI developers, researchers, and organizations use Prolific to gather data from paid study participants with a wide variety of experiences, knowledge, and skills.

The RoleWe are seeking a Queue Lead to join a high-priority initiative in partnership with a leading AI company. This is an operational, hands-on role, you will own day-to-day execution for one or more annotation queues on a live AI training/eval program: managing throughput, running quality calibrations, tracking contributor performance, and building out reporting systems that keep stakeholders informed in real time.

You will have experience running queues and quality on AI data programs. The selected candidate will be able to speak to specific projects, calibration processes they’ve built, and contributor performance systems they’ve run.

What you'll be doing

Queue Operations & Task Management

  • Own work queues: intake, triage, prioritization, task assignment, and SLA delivery
  • Monitor task queue and backlog control
  • Identify and troubleshoot bottlenecks (unclear policy, tooling issues, supply gaps)

Quality Assurance & Performance Management

  • Monitor quality at project and individual annotator or contributor level.
  • Perform QA tasks such as making corrections, providing annotator feedback, flagging systematic confusion and suggestions to team lead.
  • Manage annotator or contributor performance by performing tasks like unassigning underperformers, identifying and retaining high-performers.

Reporting & Dashboards

  • Build and maintain dashboards to track throughput, disagreement rates, defect or rework rates, and contributor performance.
  • Turn raw queue data into clear, actionable reporting for stakeholders.
  • Provide feedback for process improvements based on data trends.

Training & Support

  • Oversee dedicated Slack channels by answering participant questions, providing support, and escalating complex issues to the Client Delivery Lead.
  • Train new annotators or contributors on project guidelines.
  • Conduct live or recorded webinars as new participants join projects.
  • Host "office hours" for participants.
  • Oversee dedicated Slack channels by answering questions, providing support, escalating complex issues to Client Delivery Lead.
What you'll bring
  • Proven, direct experience running queues and quality on AI data programs, ideally at a competitor.
  • 3+ years in high-throughput ops (data labeling, trust & safety, BPO, content ops, CX ops), with recent, specific AI data ops experience
  • Able to identify shallow, low-effort, AI-generated, or phoned-in responses.
  • Proven ability to design and run calibration processes from scratch.
  • Ability to track record by building or maintaining dashboards or reporting systems (spreadsheets is required, SQL/BI/Looker/Tableau a strong plus)
  • Experience managing contributor performance at scale including making retention or removal decisions.
  • Comfortable with ambiguity and fast-changing priorities on live programs.
What "good" looks like for us:
  • Scale up to running a full AI end-to-end data program, owning pipeline, throughput, calibration, and contributor performance at program level.
  • Operates as an AI training consultant, you are comfortable advising on processes and methodology.
Bonus
  • Experience with ML data or LLM eval workflows (ranking, rubric-based judging, gold sets, calibration)
  • Distributed team coordination across time zones; vendor or contract workforce experience
  • Consulting-style experience of advising clients on AI training methodology.

Success looks likePredictable SLA delivery, a stable backlog, improving quality and reducing rework, and proactive dashboards reporting that give stakeholders real-time visibility.

Why Prolific is a great platform to join as an expert participant

Joining our platform as a participant will give you the chance to influence the AI models of the future. Once you pass our assessment, you can join Prolific in just 15 minutes and start enjoying competitive pay rates, flexible hours, and the ability to work from home.

We've built a unique platform that connects researchers and companies with a global pool of participants, enabling the collection of high-quality, ethically sourced human behavioral data and feedback. This data is the cornerstone of developing more accurate, nuanced, and aligned AI systems.

We believe that the next leap in AI capabilities won't come solely from scaling existing models, but from integrating diverse human perspectives and behaviors into AI development. By providing this crucial human data infrastructure, Prolific is positioning itself at the frontier of the next wave of AI innovation – one that reflects the breadth and the best of humanity.

Links to more information on Prolific

Website

Youtube

Privacy Statement

By submitting your application, you agree that Prolific may collect your personal data for recruiting and global organization planning. Prolific's Candidate Privacy Notice explains what personal information Prolific may process, where Prolific may process your personal information, its purposes for processing your personal information, and the rights you can exercise over Prolific's use of your personal data.

Consigue la evaluación confidencial y gratuita de tu currículum.

o arrastra y suelta tu archivo aquí

Similar jobs

Puestos de trabajo similares que vale la pena comparar

Queue Lead
Queue Lead

Prolific • EE. UU.

A distancia
USD 55.000 - 96.000
Database Administrator - Graduates - AI Training - Chihuahua, Mexico
Database Administrator - Graduates - AI Training - Chihuahua, Mexico

Prolific • EE. UU.

A distancia
USD 28.000 - 40.000
Work from home
Customer Support Reps - AI Training - Indianapolis, US
Customer Support Reps - AI Training - Indianapolis, US

Prolific • EE. UU.

A distancia
USD 25.000 - 41.000
Customer Support Reps - AI Training - Canterbury, UK
Customer Support Reps - AI Training - Canterbury, UK

Prolific • EE. UU.

A distancia
USD 34.000 - 49.000
Customer Support Reps - AI Training - Oxford, UK
Customer Support Reps - AI Training - Oxford, UK

Prolific • EE. UU.

A distancia
USD 25.000 - 41.000
PayPal payments
Customer Support Reps - AI Training - Austin, US
Customer Support Reps - AI Training - Austin, US

Prolific • EE. UU.

A distancia
USD 34.000 - 49.000
PayPal account
Customer Support Reps - AI Training - San Francisco, US
Customer Support Reps - AI Training - San Francisco, US

Prolific • EE. UU.

A distancia
USD 25.000 - 41.000
Work from home
Flexible hours
Database Administrator - Graduates - AI Training - Durango, Mexico
Database Administrator - Graduates - AI Training - Durango, Mexico

Prolific • EE. UU.

A distancia
USD 21.000 - 34.000
PayPal payments
Database Administrator - Graduates - AI Training - Veracruz, Mexico
Database Administrator - Graduates - AI Training - Veracruz, Mexico

Prolific • EE. UU.

A distancia
USD 28.000 - 40.000
Work from home
Flexible hours
Customer Support Reps - AI Training - Washington DC, US
Customer Support Reps - AI Training - Washington DC, US

Prolific • EE. UU.

A distancia
USD 34.000 - 49.000
Flexible hours
Work from home
Competitive pay