Senior LLM Inference Engineer — Cloud & Accelerators

SAM Seamless Network

San Diego (CA)

On-site

USD 162,000 - 243,000

Full time

6 days ago
Be an early applicant

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Job summary

Qualcomm Technologies, Inc. seeks experienced software engineers to advance software for large-scale LLM inference on Qualcomm platforms. The role spans open-source LLM serving systems, distributed architectures, cloud-native tooling, and accelerator integration.

You will contribute to llm-d and related projects, collaborate with open-source communities, and help design end-to-end solutions for scalable, production-grade LLM inference on Qualcomm hardware.

Qualifications

  • Bachelor's degree or higher with 4+ years in hardware/software/systems engineering or related field; or 3+ years with Master's; or 2+ years with PhD.

Responsibilities

  • Contribute features, improvements, bug fixes, and architectural enhancements to open-source LLM inference projects (llm-d and others).
  • Develop and improve support for Qualcomm AI accelerators in open-source LLM serving frameworks.
  • Work with upstream maintainers to design and upstream Qualcomm-related functionality.
  • Develop and enhance inference engine integrations for Qualcomm platforms (device abstraction, runtime integration, memory management, distributed execution, compilation, performance).
  • Enhance llm-d integration and deployment capabilities including reference architectures and tooling.
  • Design and implement distributed LLM inference capabilities (prefill/decode disaggregation, KV-cache transfer, routing, load balancing, model parallelism, multi-node, autoscaling, fault tolerance).
  • Integrate LLM inference software with Kubernetes and cloud-native infrastructure (scheduling, autoscaling, networking, observability, lifecycle).
  • Profile and optimize end-to-end inference performance (TTFT, ITL, throughput, memory, scalability).
  • Diagnose issues across model-serving frameworks, runtimes, OS, networking, accelerator runtimes, and hardware.

Skills

Python
C++
Go
Rust
Distributed systems
Cloud-native/Kubernetes
LLM inference
Linux debugging

Education

Bachelor's degree in Computer Science, Engineering, Information Systems, or related field

Tools

llm-d
KServe
Kubernetes
cloud-native tooling

Job description

Qualcomm Technologies, Inc. seeks experienced software engineers to advance software for large-scale LLM inference on Qualcomm platforms. The role spans open-source LLM serving systems, distributed architectures, cloud-native tooling, and accelerator integration.

You will contribute to llm-d and related projects, collaborate with open-source communities, and help design end-to-end solutions for scalable, production-grade LLM inference on Qualcomm hardware.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Senior LLM Inference Engineer: Infra & Accelerators
Senior LLM Inference Engineer: Infra & Accelerators

QUALCOMM, Inc. • Santa Clara (CA)

On-site
USD 162,000 - 243,000
Senior LLM Inference Engineer — Distributed Systems
Senior LLM Inference Engineer — Distributed Systems

QUALCOMM, Inc. • Santa Clara (CA)

On-site
USD 162,000 - 243,000
Senior / Staff LLM Serving Engineer – Cloud AI Inference
Senior / Staff LLM Serving Engineer – Cloud AI Inference

SAM Seamless Network • San Diego (CA)

On-site
USD 162,000 - 243,000
Principal AI Systems Engineer - LLM Inference (Cloud)
Principal AI Systems Engineer - LLM Inference (Cloud)

Qualcomm • San Diego (CA)

On-site
USD 224,000 - 335,000
LLM Serving Engineer (Cloud AI Engineering), Senior / Staff Engineer
LLM Serving Engineer (Cloud AI Engineering), Senior / Staff Engineer

Qualcomm • San Diego (CA)

On-site
USD 158,400 - 237,600
Competitive annual discretionary bonus program
Potential RSU grants
Comprehensive benefits package
Senior/Staff Software Engineer - LLM Inference Systems & Open-Source
Senior/Staff Software Engineer - LLM Inference Systems & Open-Source

SAM Seamless Network • San Diego (CA)

On-site
USD 162,000 - 243,000
Principal SW Engineer - LLM Serving (Cloud AI)
Principal SW Engineer - LLM Serving (Cloud AI)

Qualcomm • San Diego (CA)

On-site
USD 224,000 - 335,000
LLM Serving Engineer (Cloud AI Engineering), Senior / Staff Engineer
LLM Serving Engineer (Cloud AI Engineering), Senior / Staff Engineer

SAM Seamless Network • San Diego (CA)

On-site
USD 162,000 - 243,000
LLM Serving Engineer (Cloud AI Engineering), Senior / Staff Engineer
LLM Serving Engineer (Cloud AI Engineering), Senior / Staff Engineer

QUALCOMM, Inc. • Austin (TX)

On-site
USD 158,400 - 237,600
Competitive salary
Annual discretionary bonus
RSU grants opportunity
Senior/Staff Software Engineer - LLM Inference Systems & Open-Source
Senior/Staff Software Engineer - LLM Inference Systems & Open-Source

QUALCOMM, Inc. • Santa Clara (CA)

On-site
USD 162,000 - 243,000