LLM Data Quality Engineer — Linguistics Pro

US Tech Solutions; Redmond (WA); On-site; USD 69,000 - 96,000; Full time

Job summary

US Tech Solutions in Redmond, WA seeks a Linguistic Engineer on a 12-month contract to build, maintain, and analyze datasets powering LLM features for smart glasses. You will collaborate with engineers, researchers, and data scientists to ensure data quality and reliability across pipelines.

The role emphasizes data labeling, experiments for rater quality, and developing automated QA processes using Python and SQL, with opportunities to impact model performance and product outcomes.

Qualifications

• Bachelor’s degree in linguistics or related field, or equivalent industry experience.
• Master’s degree preferred or equivalent experience.
• Experience with Coursera/Udemy courses is a plus.

Responsibilities

• Aggregate cross-functional requests and translate requests into actionable datasets via synthetic data collections and on live user traffic.
• Design and conduct experiments for evaluating rater quality and inter-rater reliability; write summary reports and present findings.
• Develop manual and automated processes for multiple concurrent projects ensuring high-quality labeled data.
• Create and perfect LLM Quality grading queues and LLM-related component quality error attribution grading queues.
• Build datasets and write guidelines in rapid fashion for new and emergent LLM data rating, creation, and annotation needs.
• Analyze system metrics such as factuality, brevity, coherence, and subcomponent performance; summarize findings and communicate results.

Skills

Python basics

SQL basics

Linguistics know-how

Data analysis

Communication skills

Education

Linguistics degree

Masters preferred

Online courses experience

Job description

US Tech Solutions in Redmond, WA seeks a Linguistic Engineer on a 12-month contract to build, maintain, and analyze datasets powering LLM features for smart glasses. You will collaborate with engineers, researchers, and data scientists to ensure data quality and reliability across pipelines.

The role emphasizes data labeling, experiments for rater quality, and developing automated QA processes using Python and SQL, with opportunities to impact model performance and product outcomes.