Senior Machine Learning Engineer

Apple, Inc.; Cupertino (CA); On-site; USD 185,000 - 325,000; Full time

Job summary

Apple is seeking a Senior/Staff Software Engineer to join the ACAI Training & Inference Platform, building scalable AI workflows for model training, fine-tuning, RL, and production inference. You will work on distributed runtimes, developer APIs, and cross-team collaboration to accelerate model development within Apple’s AI ecosystem.

The role emphasizes hands-on systems work, Python/C++/Rust/Go, and deep knowledge of distributed training/inference, aiming to reduce latency and cost while

Qualifications

• Six or more years designing distributed systems, AI platforms, or high-performance software.
• Hands-on experience in large-scale AI training, LLM post-training or RL, or LLM inference/serving optimization.
• Proficiency in Python and a systems language (C++, Rust, Go, or Java).
• Experience with PyTorch and foundation models including execution/performance needs.
• Understanding distributed execution and parallelism for large models.
• Experience profiling/optimizing AI workloads across GPUs/accelerators.
• Familiarity with Ray or similar distributed execution frameworks.
• Ability to deliver reusable APIs/SDKs for model development/evaluation/deployment.
• BS/MS/PhD in CS/AI or equivalent experience.

Responsibilities

• Design scalable distributed AI execution frameworks for training, post-training, RL, and inference using Ray and related tech.
• Enable end-to-end post-training workflows from dataset prep to distributed execution and evaluation.
• Optimize LLM inference runtimes for latency, throughput, and cost across hardware configurations.
• Evaluate and integrate modern engines like vLLM, SGLang, and others for efficient AI execution.
• Design distributed execution strategies including parallelism and resource allocation.
• Improve reliability through fault tolerance, checkpointing, and observability.
• Collaborate with AI researchers to translate workload requirements into reusable capabilities.
• Establish benchmarks to quantify training efficiency, inference latency, and cost per token.
• Contribute architectural improvements to Ray and related OSS projects.

Skills

Distributed systems

AI training & inference

Python

C++

Rust

Go

Java

PyTorch

Ray

Performance optimization

Education

B.S./M.S./Ph.D. in CS or related field

Tools

Ray

TRL

VeRL

RLlib

DeepSpeed

vLLM

SGLang

Kubernetes

FSDP

Job description

Cupertino, California, United States Software and Services

The ACAI Training & Inference Platform team is seeking a Senior / Staff Software Engineer with strong distributed systems expertise and deep experience in AI training and inference. As part of Apple Cloud AI (ACAI) within Apple’s Data Platform (ADP), you will design and build core capabilities for large-scale model training, post-training, fine-tuning, reinforcement learning, and high-performance inference.You will develop scalable AI execution frameworks, developer-facing APIs, and runtime optimizations that make it easier to train, adapt, and serve advanced models efficiently across Apple’s AI ecosystem. You will partner closely with AI researchers and engineers to translate evolving model architectures and workload requirements into reliable, high-performance training and inference solutions. This role is ideal for an engineer who combines deep distributed-systems expertise with hands-on knowledge of modern LLM training, post-training, and inference optimization.

Description

ADP brings together the Apple Cloud Data Platform (ACDP), Apple Cloud AI (ACAI), and Apple Governance Platform (AGP) to provide foundational capabilities for data, analytics, AI, and governance at Apple scale. Within ACAI, the Training & Inference Platform team enables scalable model development and production inference across Apple’s AI ecosystem.As a Senior / Staff Software Engineer, you will build capabilities spanning the model lifecycle, from distributed training and post-training to optimized inference and serving.On the training side, you will develop frameworks and APIs for fine-tuning, reinforcement learning, and post-training workflows, enabling AI teams to efficiently adapt foundation models using techniques such as supervised fine-tuning (SFT), preference optimization, and reinforcement learning from feedback or verifiable rewards. You will simplify the execution of complex training pipelines, improve experimentation and iteration speed, and enable reusable, production-ready training workflows.On the inference side, you will drive deep runtime and LLM-serving optimizations to improve throughput, reduce latency, and lower the cost of serving both real time and batch inference workloads. You will optimize execution across heterogeneous hardware and model architectures, leveraging techniques such as continuous batching, KV-cache optimization, quantization, speculative decoding, disaggregated prefill and decode, and efficient distributed inference.Across both areas, you will improve the performance, scalability, reliability, and developer experience of AI training and inference workloads. You will work closely with AI practitioners to identify bottlenecks across model execution, scheduling, communication, and resource utilization, translating these insights into end-to-end improvements in model development and serving efficiency.

Responsibilities

• Design and build scalable distributed AI execution frameworks for large-scale model training, fine-tuning, post-training, reinforcement learning, and inference using Ray and related technologies. Develop high-level APIs, SDKs, and reusable workflows that simplify model training and adaptation, including supervised fine-tuning, preference optimization, and reinforcement learning workflows.


• Enable end-to-end post-training workflows, from dataset preparation and experiment orchestration through distributed execution, evaluation, and iterative model improvement.


• Optimize LLM inference runtimes for cost, latency, throughput, and resource efficiency across diverse model architectures and hardware configurations. Drive deep runtime optimizations spanning GPU execution, memory management, kernel utilization, communication, batching, KV-cache management, and distributed inference.


• Evaluate and integrate modern inference and training engines, including vLLM, SGLang, and other relevant frameworks, to deliver efficient and extensible AI execution capabilities.


• Design and optimize distributed execution strategies for large models, including parallelism, workload placement, resource allocation, and communication efficiency.


• Improve training and inference reliability through fault tolerance, checkpointing, workload recovery, observability, and performance diagnostics.


• Partner with AI researchers, model developers, and product teams to translate model and workload requirements into reusable capabilities that accelerate model development and production adoption.


• Establish performance benchmarks and optimization strategies that quantify improvements in training efficiency, post-training iteration speed, inference latency, throughput, and cost per token.


• Contribute architectural improvements to Ray and related open-source AI systems where appropriate.

Minimum Qualifications

• 6+ years of experience building distributed systems, AI platforms, model execution frameworks, or high-performance software systems.


• Strong hands-on experience in at least one of the following areas: large-scale AI training, LLM post-training and reinforcement learning, or LLM inference and serving optimization.


• Proficiency in Python and experience with a systems-level language such as C++, Rust, Go, or Java.


• Experience with PyTorch and modern foundation model architectures, including the execution and performance requirements of large language models.


• Understanding of distributed execution and parallelism strategies for large models, including data, tensor, pipeline, or expert parallelism.


• Experience profiling and optimizing AI workloads across GPUs or other accelerators, with the ability to identify and resolve performance bottlenecks across compute, memory, and communication.


• Strong understanding of at least one of the following: distributed training and post-training workflows, reinforcement learning systems, LLM inference runtimes, or high-performance model serving.


• Experience building APIs, SDKs, or reusable software components that simplify AI model development, training, adaptation, evaluation, or deployment.


• Experience with Ray or comparable distributed execution frameworks, and familiarity with production-scale AI workloads.


• Strong software design, debugging, and performance analysis skills, with the ability to deliver reliable, scalable, production-quality systems.


• B.S., M.S., or Ph.D. in Computer Science, Artificial Intelligence, or a related technical field, or comparable software engineering experience.

Preferred Qualifications

• Experience building end-to-end LLM post-training workflows, including supervised fine-tuning (SFT), preference optimization, RLHF, or reinforcement learning with verifiable rewards (RLVR).


• Experience with distributed training and post-training frameworks such as Ray Train, TRL, VeRL, RLlib, DeepSpeed, FSDP, or equivalent technologies.


• Experience designing developer-facing training APIs or services that support iterative experimentation, checkpointing, evaluation, and scalable model adaptation.


• Deep understanding of LLM inference optimization, including continuous batching, KV-cache management, quantization, speculative decoding, prefix caching, and disaggregated prefill/decode.


• Experience optimizing inference engines such as vLLM, SGLang, TensorRT-LLM, or similar frameworks for latency, throughput, GPU utilization, and cost per token.


• Experience with deep runtime optimization, including GPU kernels, memory allocation, graph execution, communication primitives, scheduling, and accelerator utilization.


• Experience benchmarking and tuning inference workloads across model architectures, hardware configurations, concurrency levels, and latency targets.


• Understanding of LLM serving architectures, including dynamic batching, autoscaling, multi-model serving, heterogeneous hardware, and workload-aware resource allocation.


• Experience integrating training and inference engines into production AI platforms through stable APIs, reusable execution abstractions, and reliable operational workflows.


• Contributions to open-source AI training, post-training, distributed execution, inference, or model-serving projects.


• Familiarity with Ray Data, Spark, Flink, or similar data-processing technologies used in AI data preparation and training pipelines.


• Experience operating AI workloads in production across cloud or on-premises accelerator environments, including Kubernetes or comparable execution systems.

At Apple, base pay is one part of our total compensation package and is determined within a range. This provides the opportunity to progress as you grow and develop within a role. The base pay range for this role is between $184,700 and $324,800, and your base pay will depend on your skills, qualifications, experience, and location.

Apple employees also have the opportunity to become an Apple shareholder through participation in Apple’s discretionary employee stock programs. Apple employees are eligible for discretionary restricted stock unit awards, and can purchase Apple stock at a discount if voluntarily participating in Apple’s Employee Stock Purchase Plan. You’ll also receive benefits including: Comprehensive medical and dental coverage, retirement benefits, a range of discounted products and free services, and for formal education related to advancing your career at Apple, reimbursement for certain educational expenses — including tuition. Additionally, this role might be eligible for discretionary bonuses or commission payments as well as relocation. Learn more about Apple Benefits

Note: Apple benefit, compensation and employee stock programs are subject to eligibility requirements and other terms of the applicable plan or program.

Apple is an equal opportunity employer that is committed to inclusion and diversity. We seek to promote equal opportunity for all applicants without regard to race, color, religion, sex, sexual orientation, gender identity, national origin, disability, Veteran status, or other legally protected characteristics. Learn more about your EEO rights as an applicant

At Apple, we believe accessibility is a fundamental human right. You’ll find that idea reflected in everything here — in our culture, our benefits and our digital tools. By welcoming as many perspectives as possible, we help you build a career where you feel like you belong.

Learn about accessibility in Apple’s workplace

Learn about reasonable accommodations for job applicants

Apple accepts applications to this posting on an ongoing basis.