On-Device AI Inference OS Engineer

OpenAI; San Francisco (CA); On-site; USD 230,000 - 385,000; Full time

Benefits offered by this job

Medical benefits

Dental benefits

Vision benefits

401(k) matching

Parental leave

Paid PTO

Relocation help

Learning stipend

Meal programs

Job summary

OpenAI in San Francisco is seeking an Operating Systems Engineer focused on on-device inference to design, develop, and ship the OS stack for reliable, energy-efficient AI on consumer devices. You will work across OS services, inference runtimes, memory management, and scheduling, partnering with research to adapt models to device constraints and push quality while honoring power and latency targets.

This role combines hardware, software, and AI, and offers collaboration across research,

Qualifications

• Substantial hands-on experience designing, developing, and debugging OS components and services.
• Proficiency in C++ for systems development with concurrency and memory management.
• Experience optimizing inference runtimes in resource-constrained environments.
• Strong scheduling, memory management, and shared resource understanding.
• Ability to diagnose complex system behavior with measure-based improvements.
• Ability to translate research and product needs into system requirements.

Responsibilities

• Build the inference platform with OS services, interfaces, and lifecycle management.
• Fit models to device constraints through quantization and memory optimization.
• Coordinate system resources with scheduling policies for latency and power.
• Advance performance and power management with adaptive execution strategies.
• Debug across the stack using tracing and profiling for reliability.
• Measure improvements with diagnostics and automated tests on devices.
• Bring capabilities to production in collaboration with multiple teams.

Skills

C++

OS design

Inference runtimes

Scheduling

Performance

Cross-functional

Rust

Job description

OpenAI in San Francisco is seeking an Operating Systems Engineer focused on on-device inference to design, develop, and ship the OS stack for reliable, energy-efficient AI on consumer devices. You will work across OS services, inference runtimes, memory management, and scheduling, partnering with research to adapt models to device constraints and push quality while honoring power and latency targets.

This role combines hardware, software, and AI, and offers collaboration across research,