Applied Scientist – LLM Training Data

Jobtailor; Overland Park (KS); On-site; USD 120,000 - 180,000; Full time

Job summary

Jobtailor is seeking an AI Data Lead to define the end-to-end data roadmap for multilingual and multimodal systems, spanning text, speech, translation, and agentic workflows.

You will design dataset curation pipelines, annotation schemas, and evaluation datasets, while ensuring secure, compliant data operations on AWS and collaborating with ML engineers, linguists, and product teams.

Qualifications

• Master’s degree in Computer Science, ML, Data Science, Computational Linguistics, Linguistics, or Statistics.
• 4+ years’ experience in AI data, ML data ops, NLP data engineering, or LLM data workflows.
• Proficiency with Python and SQL and build dataset curation pipelines.
• Experience with annotation workflows, QA rubrics, evaluation datasets, or human-in-the-loop data processes.
• Familiarity with multilingual NLP, speech data, translation data, low-resource languages, or agentic AI datasets.

Responsibilities

• Define the end-to-end data roadmap for multilingual and multimodal AI systems.
• Design and build dataset curation pipelines for training, post-training, and evaluation.
• Create annotation schemas, labeling guidelines, QA rubrics, and reviewer workflows.
• Build evaluation datasets and analyze model failure modes for targeted data improvements.
• Support post-training data workflows such as SFT, instruction tuning, and RLHF-style data.
• Use AWS-based data infrastructure to scale secure, traceable, and compliant AI data workflows.

Skills

Python Programming

SQL Proficiency

Dataset Curation Pipelines

AWS Data Tools

Multilingual NLP

Education

Master’s degree in Computer Science

Master’s degree in Machine Learning

Master’s degree in Data Science

Master’s degree in Computational Linguistics

Master’s degree in Linguistics

Master’s degree in Statistics

Tools

AWS S3

AWS Glue

AWS SageMaker

AWS Bedrock

AWS Lambda

AWS Step Functions

AWS EKS

AWS ECS

AWS IAM

AWS KMS

Job description

• Define the end-to-end data roadmap for multilingual and multimodal AI systems, including text, speech, translation, interpretation, low-resource languages, and agentic AI workflows
• Design and build dataset curation pipelines for training, post-training, and evaluation, including cleaning, deduplication, filtering, PII redaction, quality scoring, sampling, balancing, and versioning
• Create annotation schemas, labeling guidelines, QA rubrics, golden datasets, and reviewer workflows for multilingual, speech, translation, and vision data
• Build evaluation datasets and benchmarks, analyze model failure modes, and translate performance gaps into targeted data improvements
• Support post-training data workflows such as SFT, instruction tuning, preference data, RLHF/DPO-style data, reward model data, and synthetic data generation
• Use modern annotation tools and AWS-based data infrastructure to scale secure, traceable, and compliant AI data workflows

Requirements

• Master’s degree in Computer Science, Machine Learning, Data Science, Computational Linguistics, Linguistics, Statistics, or a related field, or equivalent practical experience
• 4+ years of experience in AI data, ML data operations, NLP data engineering, applied ML, speech/translation data, or LLM data workflows
• Strong hands-on experience with Python, SQL, and dataset curation pipelines
• Experience with annotation workflows, QA rubrics, evaluation datasets, or human-in-the-loop data processes
• Familiarity with multilingual NLP, speech data, translation data, low-resource languages, conversational AI, or agentic AI datasets
• Working knowledge of AWS data and ML tools such as S3, Glue, SageMaker, Bedrock, Lambda, Step Functions, EKS/ECS, IAM, or KMS
• Strong communication skills and ability to work with ML engineers, applied scientists, product teams, linguists, data teams, and vendors

Core Competencies

Demonstrates expertise in defining data roadmaps for multilingual and multimodal AI systems, with a strong focus on dataset curation, evaluation, and annotation workflows. Proficient in leveraging AWS data infrastructure and tools to enhance AI data operations and ensure compliance.

Highest-signal resume keywords

• Python Programming
• SQL Proficiency
• Dataset Curation Pipelines
• AWS Data Tools
• Multilingual NLP

Hard Skills

• Dataset Curation
• Data Cleaning
• Data Deduplication
• Quality Scoring
• Annotation Workflows
• Evaluation Datasets
• Model Analysis
• Synthetic Data Generation
• Performance Gap Analysis
• Human-in-the-Loop Processes

Soft Skills

• Strong Communication Skills

Certifications & Qualifications

• Master’s Degree in Computer Science
• Master’s Degree in Machine Learning
• Master’s Degree in Data Science
• Master’s Degree in Computational Linguistics
• Master’s Degree in Linguistics
• Master’s Degree in Statistics

Industry Keywords

• AI Data
• ML Data Operations
• NLP Data Engineering
• Speech Data
• Translation Data
• Low-Resource Languages
• Conversational AI
• Agentic AI
• Applied ML
• Data Workflows

Tools & Technologies

• AWS S3
• AWS Glue
• AWS SageMaker
• AWS Bedrock
• AWS Lambda
• AWS Step Functions
• AWS EKS
• AWS ECS
• AWS IAM
• AWS KMS