Senior LLM Inference Systems Engineer

Snowflake; Menlo Park (CA); On-site; USD 236,000 - 310,000; Full time

Benefits offered by this job

Bonus and equity plan

Medical, dental, vision insurance

401(k) retirement plan

Paid holidays

Job summary

Snowflake is seeking talented systems developers and researchers to advance LLM inference systems and optimization. You will design high-performance, intelligent inference stacks spanning distributed serving, runtime systems, and GPU kernels, collaborating with model researchers and product teams.

The role emphasizes AI-native engineering to accelerate both model performance and inference development, with opportunities to publish innovations and contribute to production-scale AI.

Qualifications

• Bachelors in CS/EE; advanced degree preferred.
• 5+ years in LLM inference or distributed AI.
• Experience with LLM architectures and serving tradeoffs.
• Hands-on with LLM inference frameworks (e.g., vLLM, TensorRT-LLM).
• Familiarity with GPU programming (CUDA, Triton).

Responsibilities

• Design high-performance LLM inference systems across devices.
• Develop techniques to reduce latency and improve throughput.
• Explore adaptive parallelism, decoding, disaggregation, batching, KV-cache.
• Build adaptive inference systems for new models and hardware.
• Apply AI-native engineering to profiling, configuration search, and optimization.
• Identify bottlenecks and drive solutions from research to production.
• Design distributed inference across GPUs and nodes.
• Develop multi-model serving, dynamic resource management, model loading.
• Profile GPUs and operators for attention, MoE, etc.
• Collaborate with researchers, infra, product; publish innovations.
• Open-source contributions and blogs.

Skills

LLM inference

Distributed AI

GPU systems

Performance optimization

System scalability

AI-native engineering

Education

Bachelor's degree in CS/EE

Master's or PhD preferred

Tools

CUDA

Triton

TensorRT-LLM

vLLM

SGLang

cuDNN

cuBLAS

CUTLASS

Nsight

Job description

Snowflake is seeking talented systems developers and researchers to advance LLM inference systems and optimization. You will design high-performance, intelligent inference stacks spanning distributed serving, runtime systems, and GPU kernels, collaborating with model researchers and product teams.

The role emphasizes AI-native engineering to accelerate both model performance and inference development, with opportunities to publish innovations and contribute to production-scale AI.