AI Accelerator Software Principal Engineer – Runtime Library

Ampere; Santa Clara (CA); Presencial; USD 182.000 - 273.000; Tempo integral

Benefícios oferecidos por esta vaga

Premium medical insurance

401K retirement plan

Unlimited flextime and holidays

Resumo da vaga

Ampere is hiring an AI Accelerator Principal Software Engineer – Runtime Library to lead design, development and optimization of AI runtime software for Ampere accelerators. You will work across systems software, performance engineering, and AI enablement to deliver high-throughput, low-latency inference on edge and cloud platforms.

You will collaborate with hardware and software teams, optimize memory and scheduling, and ensure robust production deployments across model types and popular

Qualificações

• BS/MS/PhD in a relevant technical field.
• Experience developing user-mode drivers or runtime libraries for GPUs or accelerators in Linux/RTOS.
• Strong C/C++ and systems programming experience (memory, threading, synchronization, profiling).
• Background in AI framework enablement and collaboration with hardware teams.
• Familiar with PyTorch, llama.cpp, ONNX frameworks and runtimes.

Responsabilidades

• Build and evolve an AI Runtime Library for Ampere accelerators.
• Own end-to-end acceleration paths across SW/HW stack including inference serving, graph/IR execution, and APIs.
• Drive HW/SW co-design to improve throughput, latency, memory efficiency, and utilization.
• Contribute to AI co-processor/software enablement and align runtime with accelerator capabilities.
• Collaborate cross-functionally to integrate runtime components into Ampere platform stacks.

Conhecimentos

C/C++

Systems programming

AI framework enablement

Linux/RTOS

Formação

Bachelor's degree in CS/Engineering or related

MS degree in CS/Engineering or related

PhD in related field

Ferramentas

PyTorch

llama.cpp

ONNX

Descrição da vaga

Invent the future with us. Ampere is a semiconductor design company for a new era, leading the future of computing with an innovative approach to CPU design focused on high-performance, energy efficient AI compute.

Description

Invent the future with us. Ampere is a semiconductor design company for a new era, leading the future of computing with an innovative approach to CPU design focused on high-performance, energy efficient AI compute.
As a pioneer in the new frontier of energy efficient high-performance computing, Ampere is part of the Softbank Group of companies driving sustainable computing for AI, Cloud, and edge applications.
Join us at Ampere and work alongside a passionate and growing team.

About The Role

As an AI Accelerator Principal Software Engineer – Runtime Library, you will lead the design, development, and optimization of AI runtime software that enables multiple state-of-the-art deep learning models to run efficiently on Ampere’s deep learning accelerators. You will work at the intersection of systems software, performance engineering, and AI enablement, helping deliver high-throughput, low-latency inference and a strong foundation for future model and framework support.

What You’ll Achieve

• Build and evolve an AI Runtime Library for Ampere accelerators that supports execution, scheduling, and lifecycle management of deep learning workloads across multiple model types and popular frameworks.
• Own end-to-end acceleration paths, going deep into the full SW/HW stack—including:
  • Inference serving and integration layers
  • Compiler/runtime interfaces and graph/IR execution flows
  • Runtime library architecture (APIs, memory management, operators, execution engines)
  • Communication mechanisms and device/host orchestration
• Drive HW/SW co-design and optimization to improve:
  • Throughput (tokens/requests per second)
  • Latency (kernel execution and scheduling efficiency)
  • Memory efficiency (buffering, paging, reuse, caching)
  • Overall compute utilization and scaling behavior
• Contribute to AI co-processor/accelerator software enablement, partnering closely with hardware and systems teams to ensure runtime and kernel strategies match accelerator capabilities and constraints.
• Collaborate cross-functionally to integrate runtime components into Ampere platform stacks, ensuring robust deployment on target environments and consistent performance in production-like workloads.

About You

• BS Computer Science, Computer Engineering, Electrical Engineering, or Software Engineering or related technical field & 8 years of related experience; or MS degree & 6 years; or PhD & 3 years
• Proven experience developing user-mode drivers and/or runtime libraries for GPUs or deep learning accelerators in Linux or RTOS environments.
• Strong expertise in C/C++ and systems-level programming (memory, threading, synchronization, performance profiling).
• Demonstrated background in AI framework enablement, partnering closely with hardware and systems teams to ensure runtime and kernel strategies match accelerator capabilities and constraints.
• Frameworks:
  • PyTorch (operator/runtime integration, graph execution, correctness/performance work)
  • llama.cpp (inference/runtime execution patterns)
  • ONNX (graph handling, interoperability, execution engines)
• Strong performance engineering skills, including profiling/diagnostics and optimization of execution pipelines, data movement, and compute kernels.
• Ability to operate effectively in a collaborative environment—owning complex components while partnering with compilers, hardware, and platform teams.

What We’ll Offer

At Ampere we believe in taking care of our employees and providing a competitive total rewards package that includes base pay, cash long-term incentive, and comprehensive benefits. The full base pay range for this role is between $182,000 and $273,000, except in the San Francisco Bay Area where the range is between $195,000 and $292,000.
Our benefits include health, wellness, and financial programs that support employees through every stage of life.

Benefit Highlights Include

• Premium medical insurance, dental insurance, vision insurance, as well as income protection and a 401K retirement plan, so that you can feel secure in your health and financial future.
• Unlimited Flextime and 10+ paid holidays so that you can embrace a healthy work-life balance.
• A variety of healthy snacks, energizing espresso, and refreshing drinks to keep you fueled and focused throughout the day.

Ampere is an inclusive and equal opportunity employer and welcomes applicants from all backgrounds. All qualified applicants will receive consideration for employment without regard to race, color, national origin, citizenship, religion, age, veteran and/or military status, sex, sexual orientation, gender, gender identity, gender expression, physical or mental disability, or any other basis protected by federal, state or local law.