Principal Software Engineer, Inference

Hewlett Packard Enterprise; Fort Collins (CO); Hybrid; USD 180,000 - 250,000; Full time

Benefits offered by this job

Health and wellbeing benefits

Professional development programs

Flexible work arrangements

Job summary

Hewlett Packard Enterprise in Fort Collins seeks a senior software architect to define and own the technical architecture for LLM serving, including engine integration, batching, and distributed execution strategies.

You will partner with performance engineering teams to optimize latency and throughput, mentor engineers, and present technical direction to stakeholders in a fast-paced, flexible work environment.

Qualifications

• Requires at least 12 years of software engineering experience with a minimum of 1 year working directly on LLM inference runtimes.

Responsibilities

• Define and own the technical architecture for LLM serving, including engine integration, batching, and distributed execution strategies.
• Partner with performance engineering teams to optimize latency and throughput while mentoring engineers and presenting technical direction to stakeholders.

Skills

LLM inference

Distributed execution

Continuous batching

KV cache management

Quantized execution

GPU scheduling

Performance engineering

Architecture design

Tools

Kubernetes

Go

Python

C++

CUDA

TensorRT-LLM

vLLM

SGLang

Job description

Define and own the technical architecture for LLM serving, including engine integration, batching, and distributed execution strategies. Partner with performance engineering teams to optimize latency and throughput while mentoring engineers and presenting technical direction to stakeholders.

Requirements

Requires at least 12 years of software engineering experience with a minimum of 1 year working directly on LLM inference runtimes. Candidates must possess expert-level proficiency in Kubernetes, Go, Python, and deep knowledge of GPU memory hierarchies and inference internals.

Key Skills

• LLM inference
• vLLM
• SGLang
• TensorRT-LLM
• Kubernetes
• Go
• Python
• C++
• CUDA
• Distributed execution
• Continuous batching
• KV cache management
• Quantized execution
• GPU scheduling
• Performance engineering
• Architecture design

Benefits

• Health and wellbeing benefits
• Professional development programs
• Flexible work arrangements