Senior Speech AI Evaluation Engineer

Socket.dev; Cupertino (CA); On-site; USD 180,000 - 240,000; Full time

Job summary

Apple is seeking a role focused on evaluating speech LLMs, building datasets that stress-test models, and defining metrics to turn outputs into reliable signals. You will work with modeling, infrastructure, and product teams to ensure rigorous evaluation before shipping to users across Apple platforms.

The position emphasizes real-time speech understanding, cross-functional collaboration, and a strong foundation in data-driven assessment within a privacy-conscious framework.

Qualifications

• Bachelor's degree in a related field or equivalent practical experience.
• Experience building or working with text, speech or audio evaluation pipelines and metrics.
• Proficiency in Python and experience building data processing pipelines at scale.
• Experience curating datasets for ML evaluation or training.
• Working knowledge of statistics for model performance evaluation.
• Familiarity with LLM evaluation techniques including automated and human evaluation methods.
• Strong written and verbal communication skills.

Responsibilities

• Own the data and metrics foundation for evaluating speech LLMs across accuracy, robustness, and conversational quality.
• Build and curate evaluation datasets reflecting real usage and design metrics and automated judges.
• Collaborate with modeling, infrastructure, and product partners to ensure quick, consistent, rigorous evaluation before customer release.

Skills

Python

Data pipelines

Statistics

LLM evaluation

Communication

Education

Bachelor's degree

Tools

Spark

Pandas

Job description

Apple is seeking a role focused on evaluating speech LLMs, building datasets that stress-test models, and defining metrics to turn outputs into reliable signals. You will work with modeling, infrastructure, and product teams to ensure rigorous evaluation before shipping to users across Apple platforms.

The position emphasizes real-time speech understanding, cross-functional collaboration, and a strong foundation in data-driven assessment within a privacy-conscious framework.