ML Engineer: Automated Evaluation & Adversarial Testing

Apple, Inc.; Culver City (CA); On-site; USD 142,000 - 263,000; Full time

Job summary

Apple in Culver City, CA, seeks an ML Engineer to design and scale automated evaluation systems for AI features, including multi-turn conversations. You will build adversarial test suites to expose weaknesses and run stress tests to ensure performance under demanding conditions.

The role emphasizes rigorous measurement and cross-functional collaboration to guide product readiness. As features evolve to agentic experiences, you will shape evaluation infrastructure and toolkits used across Apple’s

Qualifications

• Bachelor’s degree in CS, ML, statistics, or related.
• 4+ years building or extending ML evaluation systems with benchmarks or quality frameworks.
• Experience defining evaluation architecture for AI/ML where the unit of analysis is conversation or session.
• Experience designing adversarial or red-teaming test methodologies for ML/AI features.
• Experience with Python and ML frameworks (PyTorch, TensorFlow, or equivalent) in production or near-production settings.
• Track record of owning technical direction for evaluation efforts across multiple features or product areas.

Responsibilities

• Define automated evaluation approach for AI features, translating quality into measurable assessments.
• Build adversarial test suites targeting model failure modes.
• Develop and execute stress test protocols for extended conversations.
• Ensure alignment between automated and human evaluation methods.
• Collaborate with engineering to integrate evaluation into development and release workflows.
• Scale adversarial test case generation and multi-turn scenario pipelines.
• Communicate evaluation findings and readiness assessments to cross-functional partners.

Skills

Python

Evaluation systems

Multi-turn eval

Cross-functional comms

Education

Bachelor’s degree in CS/ML/Statistics

Tools

LangChain

LangSmith

Arize

Braintrust

Job description

Apple in Culver City, CA, seeks an ML Engineer to design and scale automated evaluation systems for AI features, including multi-turn conversations. You will build adversarial test suites to expose weaknesses and run stress tests to ensure performance under demanding conditions.

The role emphasizes rigorous measurement and cross-functional collaboration to guide product readiness. As features evolve to agentic experiences, you will shape evaluation infrastructure and toolkits used across Apple’s