ML Evaluation Architect — Automated QA & Adversarial Testing

Apple, Inc.; San Diego (CA); Presencial; USD 142.000 - 263.000; Jornada completa

Ventajas ofrecidas por este puesto de trabajo

Apple Benefits

Employee stock programs

Relocation assistance

Tuition reimbursement

Discretionary bonuses

Descripción de la vacante

Apple in San Diego, CA is seeking an ML Engineer for Automated Evaluation and Adversarial Design to build scalable evaluation systems and stress-test AI features across multi-turn interactions.

You will define evaluation architectures, create adversarial test suites, and collaborate with engineers to integrate this into product release workflows, driving model readiness for hundreds of millions of users.

Formación

• Bachelor's degree in CS, ML, statistics, or related field.
• 4+ years building/extending ML evaluation systems and benchmarks.
• Ability to define evaluation architecture for AI/ML with conversation-level analysis.
• Experience designing adversarial / red-teaming test methodologies for ML features.
• Experience with Python and ML frameworks (PyTorch, TensorFlow, or equivalent).
• Track record owning evaluation direction across multiple features.

Responsabilidades

• Define and own automated evaluation for AI features across single-turn and multi-turn experiences.
• Build adversarial test suites targeting model failure modes in productivity workflows.
• Develop and execute stress tests for extended conversations and multi-step tasks.
• Align automated and human evaluation methods and resolve disagreements.
• Collaborate with engineering to integrate evaluation into release workflows.
• Scale adversarial test generation and stress test execution with automation.
• Communicate evaluation findings and readiness to cross-functional partners.

Conocimientos

Python

ML frameworks

Evaluation architecture

Multi-turn evaluation

Adversarial testing

Red-teaming

System design

Educación

Bachelor’s degree in Computer Science, Machine Learning, Statistics, or a related field

Herramientas

LangChain

LangGraph

CrewAI

AutoGen

LangSmith

Braintrust

Arize

Descripción del empleo

Apple in San Diego, CA is seeking an ML Engineer for Automated Evaluation and Adversarial Design to build scalable evaluation systems and stress-test AI features across multi-turn interactions.

You will define evaluation architectures, create adversarial test suites, and collaborate with engineers to integrate this into product release workflows, driving model readiness for hundreds of millions of users.