Senior ML Foundation Model Compute Infra Engineer

Apple Inc.; Cupertino (CA); On-site; USD 184,700 - 324,800; Full time

Benefits offered by this job

Stock programs

Relocation

Tuition reimbursement

Medical coverage

Job summary

Apple Inc. in Cupertino, CA is seeking a Senior/Staff Engineer for the Foundation Model Compute Infrastructure team to design and build large-scale infrastructure for foundation model training, fine-tuning, evaluation and inference.

You will develop model inference and fine-tuning services, onboard accelerators, and improve reliability, performance and scalability across Apple’s AI workloads. The role collaborates with researchers and engineers to enhance developer productivity and integrate

Qualifications

• 5+ years of experience building large-scale distributed systems or cloud infrastructure.
• Experience with distributed ML training or inference systems.
• Strong programming skills in Python, Go, C++, or similar systems languages.
• Experience with accelerator infrastructure such as TPU, GPU.
• Experience with Kubernetes, container orchestration, or large-scale cluster management systems.
• Strong communication and collaboration skills across engineering and research teams.
• Bachelor’s degree in Computer Science, Engineering, or related field.

Responsibilities

• Design, build, and evolve large-scale model serving and fine-tuning services for foundation model workloads.
• Develop reliable infrastructure for model deployment, serving autoscaling, traffic management, job execution, container orchestration, and serving performance analysis.
• Improve the performance and usability of model serving and fine-tuning workloads by optimizing latency, throughput, availability, accelerator utilization, checkpoint loading, compilation caching, KV-cache-aware routing, and workflows for launching, monitoring, debugging, evaluating, and deploying models.
• Onboard and Benchmark new accelerator technologies into Apple’s compute infrastructure.
• Collaborate with the Apple Foundation Model team to integrate technologies such as Pathways, Ray, and Beam, or expose them as reliable and scalable services.
• Mentor engineers and partner across teams to influence the technical direction of Apple’s foundation model compute infrastructure.

Skills

Python

Go

C++

Education

Bachelor's degree in Computer Science or Engineering

MS or PhD in Computer Science or related field

Tools

Kubernetes

Job description

Apple Inc. in Cupertino, CA is seeking a Senior/Staff Engineer for the Foundation Model Compute Infrastructure team to design and build large-scale infrastructure for foundation model training, fine-tuning, evaluation and inference.

You will develop model inference and fine-tuning services, onboard accelerators, and improve reliability, performance and scalability across Apple’s AI workloads. The role collaborates with researchers and engineers to enhance developer productivity and integrate