LLM Data Quality Lead

US Tech Solutions; Sunnyvale (CA); On-site; USD 90,000 - 120,000; Part time

Job summary

US Tech Solutions is seeking a Quality Analyst for Human-created Data in Sunnyvale to join our technical teams. You will liaise with modeling groups and vendor leads to ensure that data produced by expert content creators meets the highest quality standards.

Responsibilities include defining data requirements, crafting detailed instructions, iterating with vendors on small data batches, and auditing data quality across collection and analysis processes.

Qualifications

• 3+ years of experience in Technical Writing, Quality Assurance or a related field.
• Excellent writing and editing skills, with the ability to adapt to different styles and tones.
• Strong analytical and problem-solving skills.
• Ability to work independently and as part of a team.

Responsibilities

• Ensure high quality during task development.
• Define data requirements based on understanding of modeling goals and study of model loss patterns.
• Design detailed instructions for content creator teams with clear examples.
• Rapidly iterate with content-production vendors on small data batches to reach desired quality.
• Measure and demonstrate data quality during data collection.
• Drive processes to deliver benchmark datasets for assessing data quality.
• Benchmark vendor data quality against competitor models.
• Use quantitative techniques to analyze vendor-produced data to assure high quality and optimize raters.
• Audit datasets for quality issues and develop tools for accelerating qualitative analysis.
• Communicate on data quality with regular reports to stakeholders.
• Track metrics such as iteration time and vendor quality audit results.
• Author best practices and guidelines for working with vendors and data instruction design.

Skills

Data Quality

LLM processes

Education

Bachelor's degree in English, Creative Writing, Linguistics, Science, Computer Science, or related field

Job description

US Tech Solutions is seeking a Quality Analyst for Human-created Data in Sunnyvale to join our technical teams. You will liaise with modeling groups and vendor leads to ensure that data produced by expert content creators meets the highest quality standards.

Responsibilities include defining data requirements, crafting detailed instructions, iterating with vendors on small data batches, and auditing data quality across collection and analysis processes.