Senior Machine Learning Engineer, Conversational AI - Services Special Projects

Apple

Cupertino (CA)

On-site

USD 185,000 - 325,000

Full time

16 hours ago
Be an early applicant

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Job summary

Apple, Inc. in Cupertino, CA is seeking a Senior Machine Learning Engineer to own the conversational intelligence layer, including agent planning, tool-calling, and real-time LLM inference. You will work across teams to productionize conversational AI at Apple scale.

The role emphasizes latency-aware development, grounding responses with knowledge graphs, and building evaluation pipelines to maintain accuracy. Strong experience with LLMs, retrieval, and low-latency systems is required.

Qualifications

  • Master's degree in CS/ML/NLP or related field.
  • 7+ years of experience in software or machine learning engineering focusing on conversational AI, search, or LLM systems.
  • Strong programming skills in Go and Python, plus Java or Scala.
  • Hands-on experience building/ shipping conversational AI or dialogue systems.
  • Experience building LLM-powered applications in production with prompt design, tool calls, and orchestration.
  • LLM inference optimization: distillation, quantization, batching, KV-cache tuning, caching.
  • Experience deploying and serving LLMs/ML models in production with TensorRT, TorchServe, etc.
  • Experience with knowledge graphs and low-latency retrieval.
  • Proficiency with PyTorch and Hugging Face ecosystem.
  • Familiarity with AWS and Docker/Kubernetes.
  • Excellent communication and collaboration skills.

Responsibilities

  • Own the agent planning and tool-calling layer; ground responses in retrieval and knowledge graphs.
  • Design conversational flows and intent understanding using LLM- and small-model parsers.
  • Reduce latency and cost through distillation, quantization, batching, and caching; select model size.
  • Develop automated evaluation for conversational quality and CI integration.
  • Define offline metrics and collaborate on A/B tests to validate improvements.
  • Ship models/services to production with versioning, monitoring, and rollback.
  • Collaborate with research, engineers, and product teams to productionize research.

Skills

ML engineering
Go & Python
LLM in production
Knowledge graphs
AWS
Communicator

Education

Master's in CS/ML/NLP
PhD in CS/ML/NLP

Tools

vLLM
SGLang
TensorRT-LLM
ONNX Runtime
PyTorch
Hugging Face transformers

Job description

Senior Machine Learning Engineer, Conversational AI - Services Special Projects

Cupertino, California, United States Software and Services

At Apple, great ideas have a way of becoming phenomenal products, services, and customer experiences very quickly. We're building a massive, real-time conversational experience from the ground up, one that will reach users at Apple scale. It sits at the intersection of Generative AI, Information Retrieval, and Knowledge Graphs.We're looking for a senior machine learning engineer to own the conversational intelligence layer: the agent's reasoning and tool-calling, the low-latency LLM inference behind it, and the knowledge services that keep every response accurate and grounded. You'll partner closely with teams across Apple to bring it to production.

Description

This is a hands-on role for an engineer who has shipped conversational AI and LLM systems to production and cares about latency as much as model quality. You'll work closely with AI/ML scientists and engineers across the stack. You'll design multi-turn conversational experiences, build agent logic that plans and calls tools, and develop query understanding models that turn natural language into structured intent. You'll also make large and small models fast and cost-efficient enough to serve in real time.Just as importantly, you'll help make sure every response is accurate and grounded. You'll build and scale knowledge graph services that resolve, enrich, and expand user requests with low latency and also connect the system to real-time data sources so its answers stay current. Finally, you'll build the evaluation that keeps all of it honest: automated and LLM-as-judge evaluation for multi-turn conversations, built into the development workflow so regressions are caught before they reach users.

Responsibilities
  • Orchestrate tools and grounding: Own the agent's planning and tool-calling layer, including tool schemas, dynamic tool selection, and structured output generation. Ground every response in retrieval, knowledge graph, and real-time data.
  • Build conversational understanding: Design and ship conversational flows, including intent understanding, context tracking, and disambiguation, using LLM- and small-model-based query parsers that feed downstream retrieval and ranking.
  • Optimize LLM inference: Profile and reduce serving latency and cost through distillation, quantization, compact output formats, dynamic batching, KV-cache tuning, and semantic caching. Use inference runtimes such as vLLM, SGLang, TensorRT-LLM, and ONNX Runtime, and decide when a request needs a large model, a small model, or none at all.
  • Build autograded evaluation: Design and own automated evaluation for conversational and generative quality, including multi-turn rubrics, LLM-as-a-judge pipelines, judge calibration against human ratings, and error-analysis tooling. Integrate eval runs into CI.
  • Connect offline metrics to online outcomes: Define offline metrics that predict user-facing wins, and partner on the A/B tests that confirm them.
  • Ship models and services to production: Take models and services from experiment to the live request path with proper versioning, monitoring, and rollback, and own their quality, latency, and reliability after launch.
  • Collaborate cross-functionally: Work closely with research scientists, engineers, and product teams to turn recent research in conversational AI, retrieval, and inference into systems that hold up in production.
Minimum Qualifications
  • Master's degree in Computer Science, Machine Learning, NLP, or a related field
  • 7+ years of experience as a software or machine learning engineer, with a significant focus on conversational AI, search, or LLM-powered systems
  • Strong programming skills in Go and Python, plus at least one of Java or Scala
  • Hands‑on experience building and shipping conversational AI or dialogue systems: intent and slot understanding, multi‑turn context management, clarification, and disambiguation
  • Experience building LLM-powered applications in production, including prompt design, tool/function calling, structured output generation, and agentic orchestration patterns
  • Demonstrated LLM inference‑optimization experience: distillation, quantization, batching, KV-cache tuning, caching, and profiling to find real bottlenecks rather than guessing at them
  • Hands‑on experience deploying and serving LLMs or ML models directly in the production request path, using serving frameworks such as vLLM, SGLang, Triton, or TorchServe
  • Experience with knowledge graphs and low‑latency lookup and retrieval services
  • Fluency with PyTorch and the Hugging Face ecosystem (transformers, PEFT)
  • Familiarity with cloud environments (including AWS) and containerization (Docker, Kubernetes)
  • Excellent communication skills and a collaborative mindset
Preferred Qualifications
  • Ph.D. in Computer Science, Machine Learning, NLP, or a related field
  • Experience fine‑tuning and distilling LLMs for task‑specific use, such as query parsing, classification, and response generation, using techniques such as LoRA/adapters and teacher/student setups
  • Experience designing knowledge graph schemas, building graph compilation and ingestion pipelines, and working with graph databases such as TigerGraph or Neo4j
  • Deep experience with KV stores and in‑memory serving for low‑latency lookups (e.g., Cassandra, Redis, SSTables)
  • Experience calibrating LLM judges against human judgments for multi‑turn conversations, including inter‑annotator agreement and judge reliability tracking
  • Experience with personalization and memory in conversational systems, and applying privacy principles such as data minimization and on‑device processing
  • Experience implementing safety guardrails for LLM‑powered agents, including prompt‑injection defenses and adversarial evaluation
  • Familiarity with experiment tracking and data/model versioning tools (MLflow, Weights & Biases, DVC)
  • Deep understanding of distributed systems and service design, and experience architecting and leading projects with cross‑functional stakeholders
  • Published work or patents in conversational AI, information retrieval, knowledge graphs, or efficient inference

At Apple, base pay is one part of our total compensation package and is determined within a range. This provides the opportunity to progress as you grow and develop within a role. The base pay range for this role is between $184,700 and $324,800, and your base pay will depend on your skills, qualifications, experience, and location.

Apple employees also have the opportunity to become an Apple shareholder through participation in Apple’s discretionary employee stock programs. Apple employees are eligible for discretionary restricted stock unit awards, and can purchase Apple stock at a discount if voluntarily participating in Apple’s Employee Stock Purchase Plan. You’ll also receive benefits including: Comprehensive medical and dental coverage, retirement benefits, a range of discounted products and free services, and for formal education related to advancing your career at Apple, reimbursement for certain educational expenses — including tuition. Additionally, this role might be eligible for discretionary bonuses or commission payments as well as relocation. Learn more about Apple Benefits

Note: Apple benefit, compensation and employee stock programs are subject to eligibility requirements and other terms of the applicable plan or program.

Apple is an equal opportunity employer that is committed to inclusion and diversity. We seek to promote equal opportunity for all applicants without regard to race, color, religion, sex, sexual orientation, gender identity, national origin, disability, Veteran status, or other legally protected characteristics. Learn more about your EEO rights as an applicant

At Apple, we believe accessibility is a fundamental human right. You’ll find that idea reflected in everything here — in our culture, our benefits and our digital tools. By welcoming as many perspectives as possible, we help you build a career where you feel like you belong. Learn about accessibility in Apple’s workplace

Learn about reasonable accommodations for job applicants

Apple accepts applications to this posting on an ongoing basis.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Lead, Conversational AI & Knowledge Intelligence
Lead, Conversational AI & Knowledge Intelligence

Apple Inc. • Santa Clara (CA)

Hybrid
USD 268,000 - 402,000
Medical and dental
Retirement benefits
Stock options
+1
Software Engineering (AI/ML), Retail and Marcom Engineering
Software Engineering (AI/ML), Retail and Marcom Engineering

Apple, Inc. • Sunnyvale (CA)

On-site
USD 150,000 - 278,000
Stock awards
Relocation assistance
Tuition reimbursement
+1
Software Engineering (AI/ML), Retail and Marcom Engineering
Software Engineering (AI/ML), Retail and Marcom Engineering

Apple Inc. • Sunnyvale (CA)

On-site
USD 150,000 - 278,000
ML Data & Eval Engineer, Siri Agent Modeling
ML Data & Eval Engineer, Siri Agent Modeling

Apple • Cupertino (CA)

On-site
USD 150,000 - 278,000
Comprehensive medical and dental
Retirement benefits
Employee stock programs
+1
Machine Learning Engineer, Answers Knowledge and Information - Proactive
Machine Learning Engineer, Answers Knowledge and Information - Proactive

Apple • Seattle (WA)

On-site
USD 175,000 - 309,000
Stock options
Relocation assistance
Comprehensive benefits
+1
Software Engineering Manager, AIML — LLM Inference Quality
Software Engineering Manager, AIML — LLM Inference Quality

Apple, Inc. • Seattle (WA)

On-site
USD 196,000 - 338,000
Medical & dental coverage
Stock programs & RSUs
Education reimbursement
+1
Machine Learning Engineer, Natural Language Understanding, Proactive
Machine Learning Engineer, Natural Language Understanding, Proactive

Apple Inc. • Santa Clara (CA)

On-site
USD 150,000 - 278,000
Machine Learning Engineer – LLMs, Agent Systems, and Simulation Tooling, Siri Core Modeling
Machine Learning Engineer – LLMs, Agent Systems, and Simulation Tooling, Siri Core Modeling

Apple, Inc. • Cupertino (CA)

On-site
USD 150,000 - 278,000
Medical and dental coverage
Employee stock programs
Educational reimbursement (tuition)
Machine Learning Research Engineer, NLP, Input Experience
Machine Learning Research Engineer, NLP, Input Experience

Apple, Inc. • Cupertino (CA)

On-site
USD 185,000 - 325,000
Comprehensive medical and dental cover
Retirement benefits
Discounted products and free services
+2
Senior ML Engineer - On-Device AI & Retrieval
Senior ML Engineer - On-Device AI & Retrieval

Apple Inc. • Cupertino (CA)

Hybrid
USD 150,000 - 278,000