Senior Speech AI Engineer: LLM Evaluation & Datasets

Apple Inc.; Cambridge (MA); Hybrid; USD 166,000 - 292,000; Full time

Benefits offered by this job

Stock options

Medical and dental coverage

Tuition reimbursement

Relocation assistance

Employee stock purchase plan

Job summary

Apple Inc. Cambridge is seeking a Sr. Machine Learning Engineer to lead evaluation of audio LLMs in the Speech domain.

You will design datasets, craft metrics, and build scalable pipelines to stress-test personalized and multilingual ASR/TTs, integrating with modeling and product teams. The role emphasizes rigorous evaluation, collaboration with infrastructure, and translating results for stakeholders. Join a team advancing how Apple measures and improves listening and conversational AI across

Qualifications

• Bachelor's degree or equivalent practical experience.
• Experience building or working with text, speech or audio evaluation pipelines and metrics.
• Proficiency in Python and building data processing pipelines at scale.
• Experience curating or annotating datasets for ML evaluation or training.
• Working knowledge of statistics for evaluating model performance.
• Familiarity with LLM evaluation techniques including automated and human evaluation.
• Strong written and verbal communication to explain results to technical and non-technical audiences.

Responsibilities

• Designs and curates audio evaluation datasets representing real-world usage, including personalized, multilingual, and conversational scenarios.
• Defines and implements evaluation metrics for audio LLMs across accuracy, robustness, and conversational/generation quality.
• Builds automated evaluation pipelines and LLM-as-judge tooling to scale audio model assessment.
• Analyzes evaluation results to identify gaps, regressions, and opportunities for improvement.
• Partners with human-evaluation programs to design rating protocols and validate automated metrics against human judgment.
• Collaborates with infrastructure teams to integrate new evaluation sets into shared tooling.
• Contributes evaluation methodology for new audio LLM capabilities as models evolve.

Skills

Python

Evaluation pipelines

Dataset curation

Statistics

LLM evaluation methods

Communication

Spark

Education

Bachelor's degree

Tools

Spark

Job description

Apple Inc. Cambridge is seeking a Sr. Machine Learning Engineer to lead evaluation of audio LLMs in the Speech domain.

You will design datasets, craft metrics, and build scalable pipelines to stress-test personalized and multilingual ASR/TTs, integrating with modeling and product teams. The role emphasizes rigorous evaluation, collaboration with infrastructure, and translating results for stakeholders. Join a team advancing how Apple measures and improves listening and conversational AI across