Senior Principal: LLM Inference & GPU Optimization

People in AI; United States; Remoto; USD 230.000 - 280.000; Tempo pieno

Vantaggi offerti da questo lavoro

Bonus

Equity

Remote work within United States

Descrizione del lavoro

People In AI is hiring a Senior Principal Software Engineer to optimize production LLM inference systems. You will work directly with inference runtimes, CUDA kernels, GPU memory architecture, and serving infrastructure for large-scale commercial applications.

This role focuses on making models run faster and more cost-efficient, going beyond deployment to optimize runtime internals, scheduling, and memory behavior. Remote work is supported for candidates in the United States.

Competenze

• Significant software engineering experience with strong C++ fundamentals.
• Hands-on experience developing, modifying, or optimizing CUDA or similar GPU kernels.
• Experience with production LLM inference systems at meaningful scale.
• Deep understanding of GPU memory hierarchy, bandwidth, and bottlenecks.
• Experience profiling and improving latency, throughput, or GPU utilization.

Mansioni

• Profile and optimize production LLM inference workloads for latency, throughput, and cost.
• Modify and improve inference runtimes such as vLLM and TensorRT LLM.
• Develop and optimize CUDA kernels and memory operations.
• Improve KV cache management, memory allocation, and inference performance under concurrency.
• Explore techniques like quantization and disaggregated inference; benchmark results in production.
• Collaborate with AI researchers and systems engineers across a global organization.

Conoscenze

C++ programming

CUDA kernels

LLM inference systems

GPU memory architecture

Performance profiling

Strumenti

vLLM

TensorRT LLM

CUDA

Descrizione del lavoro

People In AI is hiring a Senior Principal Software Engineer to optimize production LLM inference systems. You will work directly with inference runtimes, CUDA kernels, GPU memory architecture, and serving infrastructure for large-scale commercial applications.

This role focuses on making models run faster and more cost-efficient, going beyond deployment to optimize runtime internals, scheduling, and memory behavior. Remote work is supported for candidates in the United States.