Principal AI Accelerator Runtime Engineer

Ampere; Santa Clara (CA); On-site; USD 182,000 - 273,000; Full time

Benefits offered by this job

Premium medical insurance

401K retirement plan

Unlimited flextime and holidays

Job summary

Ampere is hiring an AI Accelerator Principal Software Engineer – Runtime Library to lead design, development and optimization of AI runtime software for Ampere accelerators. You will work across systems software, performance engineering, and AI enablement to deliver high-throughput, low-latency inference on edge and cloud platforms.

You will collaborate with hardware and software teams, optimize memory and scheduling, and ensure robust production deployments across model types and popular

Qualifications

• BS/MS/PhD in a relevant technical field.
• Experience developing user-mode drivers or runtime libraries for GPUs or accelerators in Linux/RTOS.
• Strong C/C++ and systems programming experience (memory, threading, synchronization, profiling).
• Background in AI framework enablement and collaboration with hardware teams.
• Familiar with PyTorch, llama.cpp, ONNX frameworks and runtimes.

Responsibilities

• Build and evolve an AI Runtime Library for Ampere accelerators.
• Own end-to-end acceleration paths across SW/HW stack including inference serving, graph/IR execution, and APIs.
• Drive HW/SW co-design to improve throughput, latency, memory efficiency, and utilization.
• Contribute to AI co-processor/software enablement and align runtime with accelerator capabilities.
• Collaborate cross-functionally to integrate runtime components into Ampere platform stacks.

Skills

C/C++

Systems programming

AI framework enablement

Linux/RTOS

Education

Bachelor's degree in CS/Engineering or related

MS degree in CS/Engineering or related

PhD in related field

Tools

PyTorch

llama.cpp

ONNX

Job description

Ampere is hiring an AI Accelerator Principal Software Engineer – Runtime Library to lead design, development and optimization of AI runtime software for Ampere accelerators. You will work across systems software, performance engineering, and AI enablement to deliver high-throughput, low-latency inference on edge and cloud platforms.

You will collaborate with hardware and software teams, optimize memory and scheduling, and ensure robust production deployments across model types and popular