Staff Experience Designer, AI Evaluation Platform

Apple Inc.; Seattle (WA); On-site; USD 185,000 - 325,000; Full time

Benefits offered by this job

Medical coverage

Dental coverage

Stock programs

Tuition reimbursement

Relocation assistance (possible)

Job summary

Apple Inc. in New York, NY, seeks a Staff Experience Designer for the AI Evaluation Platform to own the end-to-end experience. You’ll set direction, craft interfaces for evaluation of non-deterministic AI systems, and partner with researchers to surface methods in product surfaces.

You’ll lead design for dashboards, uncertainty visualization, and multi-surface workflows, while building a scalable design system with cross-functional teams.

Qualifications

• Minimum 8+ years designing digital products or experiences for technical users, with end-to-end ownership.
• Portfolio with at least one case study showing problem framing to shipped outcome.
• Ability to design complex information: dashboards, comparisons, and uncertain results.
• Strong interaction and visual design craft for dense interfaces.
• Fluency in AI/ML concepts to work with researchers/engineers.
• Familiarity with AI tools to build working prototypes.
• Excellent communication to gain buy-in across teams.

Responsibilities

• Design the platform experience end to end for evaluating AI systems.
• Make dense results legible: scores, distributions, and uncertainty.
• Define inspection of multi-step agent behavior and root-cause analysis.
• Collaborate with researchers to surface techniques in product interfaces.
• Build and maintain the platform's design system as first designer.
• Prototype with real data to validate ideas before build.
• Run research directly with engineers, scientists, and practitioners.

Skills

End-to-end ownership

Case study portfolio

Dashboards & data visuals

Interaction design

AI/ML concepts familiarity

Prototyping with AI tools

Communication & collaboration

Tools

LangSmith

Braintrust

D3

Vega-Lite

SQL

Notebooks

Job description

Staff Experience Designer, AI Evaluation Platform

New York City, New York, United States Software and Services

AI systems are only as trustworthy as the methods used to evaluate them. At Apple, where AI powers experiences for billions of people, getting evaluation right is not a support function, it is a foundational science. Our team, part of Apple Services Engineering, builds the platform that teams across Apple use to evaluate the AI and agentic systems they ship. It's where they define what "good" means, prove it, and act on what they find. Evaluating non-deterministic systems is one of the hardest unsolved problems in production ML, and one Apple has to get right at scale. We're looking for a Staff Experience Designer to own that experience end to end. You'll be the first designer on this platform, and evaluation as a design practice doesn't have settled patterns yet, so your work will help define them.

Description

Teams building AI features need to know if what they shipped works. Today that means navigating unfamiliar territory: scoring something non-deterministic, telling a real regression from noise, trusting a judgment a model made instead of a person. Most existing tools here were built for the researchers who invented these methods, not for the people who now need to use them daily. You'll set the direction here and build the design system everyone else builds on. We'd rather get a rough version in front of real users than a polished one later. And because teams use this platform to decide what to ship, trust matters more than usual: provenance, uncertainty, and edge cases determine whether someone acts on a result or quietly stops believing it. What makes this team unusual is its interdisciplinary core. Alongside the platform, we run a research group working on evaluation methodology itself, including how to tell whether an evaluator is calibrated, biased, or measuring what it claims to. Their methods ship into this platform, and you decide how anyone first encounters them. You'll be close to that work while it's still forming, instead of picking it up once it's finished.

Responsibilities

• Design the path from a vague question about a system’s quality or safety to a running evaluation, without requiring evaluation expertise.
• Make dense, multi-dimensional results legible enough to act on: scores, comparisons, distributions, and the uncertainty attached to them.
• Define how someone inspects multi-step agent behavior, traces a failure to its cause, and audits a judgment a model made.
• Partner with our research scientists as new evaluation techniques are being developed, so you shape how they surface in the product instead of designing around them after they're set. You'll translate these techniques into interfaces non-specialists can use and trust.
• Build and maintain the platform's design system as the first designer on the team.
• Prototype against real data to validate ideas before they're built.
• Run research directly with the engineers, scientists, and practitioners who use the platform.

Minimum Qualifications

• 8+ years of experience designing digital products or experiences, including end-to-end ownership of complex, data-dense products for technical users. You've set direction in spaces with no existing precedent, and you get to clarity by running research yourself with the people who use what you build.
• A portfolio you can share, including at least one case study that walks through your process from problem framing to shipped outcome.
• Proven ability to design complex information: dashboards, comparison views, large result sets, and results that carry statistical uncertainty.
• Strong interaction and visual design skills, with a high bar for craft in dense, information-heavy interfaces.
• Enough fluency in AI/ML concepts (benchmarks, metrics, model-based judging, agentic systems) to work directly with a research scientist or engineer.
• Fluency with AI tools in your own practice. You use Claude Code or equivalents to build working prototypes and extend what you can make on your own.
• Excellent communication skills, with the ability to build buy-in across engineering and research without formal authority.

Preferred Qualifications

• Experience as the first or only designer on a platform, and/or experience translating research output into shipped product.
• Experience building and maintaining a design system, including its components, patterns, and naming conventions.
• An AI-first instinct for interface design: shipped interfaces organized around stated intent, or surfaces meant to be operated by agents as well as people.
• Experience designing coherent workflows across multiple surfaces (UI, CLI, SDK) that need to stay consistent with each other.
• Familiarity with modern evaluation, observability, or visualization tooling (e.g. LangSmith, Braintrust, D3, Vega-Lite).
• Comfort with SQL and notebooks to explore your own data.

At Apple, base pay is one part of our total compensation package and is determined within a range. This provides the opportunity to progress as you grow and develop within a role. The base pay range for this role is between $184,700 and $324,800, and your base pay will depend on your skills, qualifications, experience, and location.
Apple employees also have the opportunity to become an Apple shareholder through participation in Apple’s discretionary employee stock programs. Apple employees are eligible for discretionary restricted stock unit awards, and can purchase Apple stock at a discount if voluntarily participating in Apple’s Employee Stock Purchase Plan. You’ll also receive benefits including: Comprehensive medical and dental coverage, retirement benefits, a range of discounted products and free services, and for formal education related to advancing your career at Apple, reimbursement for certain educational expenses — including tuition. Additionally, this role might be eligible for discretionary bonuses or commission payments as well as relocation. Learn more about Apple Benefits

Note: Apple benefit, compensation and employee stock programs are subject to eligibility requirements and other terms of the applicable plan or program.

Apple is an equal opportunity employer that is committed to inclusion and diversity. We seek to promote equal opportunity for all applicants without regard to race, color, religion, sex, sexual orientation, gender identity, national origin, disability, Veteran status, or other legally protected characteristics. Learn more about your EEO rights as an applicant

At Apple, we believe accessibility is a fundamental human right. You’ll find that idea reflected in everything here — in our culture, our benefits and our digital tools. By welcoming as many perspectives as possible, we help you build a career where you feel like you belong.
Learn about accessibility in Apple’s workplace
Learn about reasonable accommodations for job applicants

Apple accepts applications to this posting on an ongoing basis.