Cloud Inference Engineer

Luminal; San Francisco (CA); On-site; USD 150,000 - 250,000; Full time

Benefits offered by this job

Equity 0.15%-0.75%

On-site in SF

Job summary

Luminal (YC S25) in San Francisco seeks a founding on-site engineer to ship low-latency, high-throughput model serving on Luminal Cloud.

You will deploy and tune models with KV caching, paged attention, and batching; lead performance reviews; improve scheduler and autoscaling; and occasionally write kernels.

This full-time role offers an annual base salary of $150,000–$250,000 USD plus equity between 0.15% and 0.75% and requires US visa/citizenship.

Qualifications

• Experience optimizing CUDA and GPU inference pipelines.
• Familiarity with vLLM, SGLang or TensorRT-LLM.
• Knowledge of KV caching, paged attention, batching and token streaming.
• Distributed GPU compute is a strong plus.

Responsibilities

• Deploy and tune models with optimizations like KV caching, paged attention, sequence packing, etc.
• Conducting model performance reviews
• Improve scheduler, batcher, autoscaling; profile latency, cost, utilization
• Sometimes write kernels and, yes, occasional tasteful shitposting

Skills

CUDA optimization

GPU inference optimization

Distributed compute

Education

No degree required

Tools

vLLM

SGLang

TensorRT-LLM

KV caching

Paged attention

Batching

Token streaming

Job description

Qualifications

• CUDA + GPU inference optimization

• vLLM, SGLang, or TensorRT-LLM experience

• KV caching, paged attention, batching, token streaming, etc.

• Distributed compute (with GPUs is a super plus)

• No degree required

Company

Luminal (YC S25) builds an AI compiler and serving stack that makes models 10x faster and production ready with one line.

Role

Founding, on site in downtown SF. Ship low latency, high throughput model serving on Luminal Cloud.

Day to day responsibilities:

• Deploy and tune models with optimizations like KV caching, paged attention, sequence packing, etc.

• Conducting model performance reviews

• Improve scheduler, batcher, autoscaling; profile latency, cost, utilization

• Sometimes write kernels and, yes, occasional tasteful shitposting

Location and compensation

San Francisco, CA, US · Full-time · On-site. Annual base salary: $150,000–$250,000 USD. Equity: 0.15%–0.75%. Visa: US citizen/visa only.