Autograder AI Scientist: GenAI Quality & Evaluation

Socket.dev; Cupertino (CA); On-site; USD 180,000 - 240,000; Full time

Job summary

Apple in Cupertino, CA is seeking an Applied AI Scientist to develop cutting-edge autograding models for evaluating GenAI features across Apple products.

You will design state-of-the-art autograders, apply prompting techniques and model adaptation, and collaborate with product, annotation and data science teams to deploy scalable evaluation tools. The role emphasizes reliability, safety, and measurable quality improvements through rigorous testing.

Qualifications

• Extensive experience with prompting techniques.
• 1+ year of industry experience in building or evaluating GenAI models.
• Familiarity with LLMOps for deploying and monitoring AI models.
• MS/PhD in Computer Science, Machine Learning, AI, or related field.
• Ownership mindset and willingness to take on tasks across teams.

Responsibilities

• Design and advance autograder systems to evaluate AI product quality at scale.
• Apply prompting and foundation-model adaptation to build robust evaluation tools.
• Collaborate with product, annotation, and data science teams to deploy autograders.
• Ensure trustworthiness, safety, and extensibility of evaluation tools.

Skills

Prompting techniques

GenAI model development

LLMOps in production

Data quality assessment

Cross-team communication

Education

MS/PhD in CS/ML/AI or related

Job description

Apple in Cupertino, CA is seeking an Applied AI Scientist to develop cutting-edge autograding models for evaluating GenAI features across Apple products.

You will design state-of-the-art autograders, apply prompting techniques and model adaptation, and collaborate with product, annotation and data science teams to deploy scalable evaluation tools. The role emphasizes reliability, safety, and measurable quality improvements through rigorous testing.