Senior Speech LLM Evaluation Engineer — Real-Time Audio AI

Apple Inc.; Cupertino (CA); Hybrid; USD 190,000 - 325,000; Full time

Benefits offered by this job

Medical and dental coverage

Apple stock programs

Tuition reimbursement

Job summary

Apple Inc. seeks a Sr. Machine Learning Engineer to advance speech LLM evaluation in the Siri organization.

You will design evaluation datasets and metrics for real-world multilingual audio use, and build scalable pipelines with LLM-as-judge tools to ensure robust model quality before user impact. You will collaborate with modeling, infrastructure, and product partners, driving rigorous evaluation and transparent communication of results to non-technical stakeholders, while aligning with Apple’s

Qualifications

• Bachelor's degree in Computer Science, Electrical Engineering, or related field, or equivalent practical experience.
• Experience building or working with text, speech or audio evaluation pipelines and metrics.
• Proficiency in Python and building data processing pipelines at scale.
• Experience curating or annotating datasets for machine learning evaluation or training.
• Working knowledge of statistics as applied to measuring model performance and interpreting evaluation results.
• Familiarity with large language model evaluation techniques, including automated (LLM-as-judge) and human evaluation methods.
• Strong written and verbal communication skills.

Responsibilities

• Designs and curates audio evaluation datasets that represent real-world usage.
• Defines and implements evaluation metrics for audio LLMs across accuracy, robustness, and conversational/generation quality.
• Builds automated evaluation pipelines and LLM-as-judge tooling to scale audio model assessment.
• Analyzes evaluation results to identify accuracy gaps, regressions, and opportunities for hillclimbing; communicates findings to modeling teams.
• Partners with human-evaluation programs to design rating protocols and validate metrics.
• Collaborates with infrastructure teams to integrate evaluation sets and metrics into shared tooling.
• Contributes evaluation methodology for new audio LLM capabilities as they emerge.

Skills

Python

Data pipelines

Dataset curation

Statistics

LLM evaluation

Communication

Education

Bachelor's degree in CS/EE or equivalent

Tools

Spark

Job description

Apple Inc. seeks a Sr. Machine Learning Engineer to advance speech LLM evaluation in the Siri organization.

You will design evaluation datasets and metrics for real-world multilingual audio use, and build scalable pipelines with LLM-as-judge tools to ensure robust model quality before user impact. You will collaborate with modeling, infrastructure, and product partners, driving rigorous evaluation and transparent communication of results to non-technical stakeholders, while aligning with Apple’s