Staff Engineer - AI Inference Infrastructure

Digitailor; San Francisco (CA); On-site; USD 200,000 - 300,000; Full time

Job summary

Digitailor in San Francisco, CA, is seeking an engineer to build AI infrastructure for open-source model inference. You will work across models, serving systems, kernels, and production infrastructure to deliver reliable, cost-efficient inference across diverse hardware.

You will own day-zero model integration, optimize latency and throughput, implement batching, KV caching, and quantization, and write CUDA/HIP/Triton kernels while collaborating with customers and the engineering team.

Qualifications

• 1–6 years in systems, backend, or infra engineering.
• Strong CS fundamentals and experience at a high-caliber tech org or startup.
• Familiarity with CUDA, HIP, Triton or GPU tooling; inference optimization a plus.

Responsibilities

• Deliver day-zero support for new open-source models and tune for latency/throughput.
• Optimize serving with batching, KV caching, speculative decoding, and quantization.
• Write and tune CUDA/HIP/Triton kernels across accelerator platforms.
• Design, deploy, and operate heterogeneous clusters across providers.
• Run production inference on mixed hardware fleets; improve reliability and cost per token.

Skills

Systems/Backend engineering

Infrastructure

Low-level programming

GPU programming

Python

Tools

CUDA

HIP

Triton

Job description

Digitailor in San Francisco, CA, is seeking an engineer to build AI infrastructure for open-source model inference. You will work across models, serving systems, kernels, and production infrastructure to deliver reliable, cost-efficient inference across diverse hardware.

You will own day-zero model integration, optimize latency and throughput, implement batching, KV caching, and quantization, and write CUDA/HIP/Triton kernels while collaborating with customers and the engineering team.