Real-Time GPU Optimization Engineer - Inference

techire ai; San Francisco (CA); On-site; USD 230,000 - 300,000; Full time

Job summary

techire ai is seeking a GPU Optimisation Engineer for real-time inference in production AI workloads. The role focuses on pushing GPU performance to sub-50ms latency under high concurrency, close to the metal across kernel and runtime layers.

You will profile and optimize large generative models, write custom CUDA/Triton kernels, and collaborate with research to deliver production-ready inference at scale. SF relocation/visa sponsorship available.

Qualifications

• Strong experience with CUDA and/or Triton.
• Deep understanding of GPU execution (memory hierarchy, scheduling, occupancy, concurrency).
• Experience optimising inference latency and throughput for large generative models.
• Familiarity with attention kernels, decoding paths, or LLM‑style runtimes.
• Comfort profiling with low‑level GPU tooling.

Responsibilities

• Profiling GPU bottlenecks across memory bandwidth, kernel fusion, quantisation, and scheduling.
• Writing and tuning custom CUDA / Triton kernels for performance‑critical paths.
• Improving attention, decoding, and KV cache efficiency in inference runtimes.
• Modifying and extending vLLM‑style systems to better suit real‑time workloads.
• Optimising models to fit GPU memory constraints without degrading output quality.
• Benchmarking across NVIDIA GPUs (with exposure to AMD and other accelerators over time).
• Partnering directly with research to turn new model ideas into fast, production‑ready inference.

Skills

CUDA

Triton

GPU architecture

Profiling tooling

Low-level optimization

Job description

techire ai is seeking a GPU Optimisation Engineer for real-time inference in production AI workloads. The role focuses on pushing GPU performance to sub-50ms latency under high concurrency, close to the metal across kernel and runtime layers.

You will profile and optimize large generative models, write custom CUDA/Triton kernels, and collaborate with research to deliver production-ready inference at scale. SF relocation/visa sponsorship available.