Senior LLM Inference Engineer: Infra & Accelerators

QUALCOMM, Inc.; Santa Clara (CA); On-site; USD 162,000 - 243,000; Full time

Job summary

Qualcomm Technologies, Inc. is seeking experienced software engineers to advance the software ecosystem for large-scale LLM inference on Qualcomm platforms.

You will work at the intersection of open-source LLM serving systems, distributed systems, and cloud-native infrastructure, contributing to llm-d and related engines to run efficiently on Qualcomm accelerators. You may focus on distributed LLM serving or inference runtime and accelerator integration while collaborating across both areas.

Qualifications

• Bachelor's degree in Computer Science, Engineering, Information Systems, or related field and 4+ years of Hardware Engineering, Software Engineering, Systems Engineering, or related work experience.
• Master's degree in Computer Science, Engineering, Information Systems, or related field and 3+ years of Hardware Engineering, Software Engineering, Systems Engineering, or related work experience.
• PhD in Computer Science, Engineering, Information Systems, or related field and 2+ years of Hardware Engineering, Software Engineering, Systems Engineering, or related work experience.
• Strong software engineering and debugging skills.
• Experience developing production-quality software in one or more of Python, C++, Go, or Rust.
• Strong understanding of ML/inference systems, distributed systems, AI accelerator software, cloud-native/Kubernetes infrastructure, ML compiler/runtime systems.

Responsibilities

• Contribute features, improvements, bug fixes, and architectural enhancements to open-source LLM inference projects, including llm-d and other open-source LLM inference and serving projects.
• Develop and improve support for Qualcomm AI accelerators in open-source LLM serving frameworks.
• Work with upstream maintainers and open-source communities to design, review, and upstream Qualcomm-related functionality.
• Develop and enhance inference engine integrations for Qualcomm platforms, including device abstraction, accelerator runtime integration, memory management, distributed execution, compilation, and performance optimization.
• Develop and improve llm-d integration and deployment capabilities for Qualcomm platforms, including reference architectures, deployment blueprints, configuration, and operational tooling.
• Design and implement distributed LLM inference capabilities such as prefill/decode disaggregation, KV-cache management, distributed communication, load balancing, model parallelism, multi-node and scale-out inference, autoscaling and resource management.
• Integrate LLM inference software with Kubernetes and cloud-native infrastructure, including scheduling, autoscaling, networking, observability, and lifecycle management.
• Profile, benchmark, and optimize end-to-end inference performance including TTFT, ITL, throughput, memory utilization, and scalability.

Skills

Python

C++

Go

Rust

Distributed systems

LLM inference

Cloud-native/Kubernetes

Education

Bachelor's degree in CS/Engineering

Master's degree in CS/Engineering

PhD in CS/Engineering

Tools

llm-d

KServe

Kubernetes

Torch

Job description

Qualcomm Technologies, Inc. is seeking experienced software engineers to advance the software ecosystem for large-scale LLM inference on Qualcomm platforms.

You will work at the intersection of open-source LLM serving systems, distributed systems, and cloud-native infrastructure, contributing to llm-d and related engines to run efficiently on Qualcomm accelerators. You may focus on distributed LLM serving or inference runtime and accelerator integration while collaborating across both areas.