Senior AI Inference Engineer (Remote)

Zoom; Seattle (WA); Hybrid; USD 207,000 - 452,000; Full time

Job summary

Zoom is seeking an experienced software engineer to design and own inference systems powering its AI models at production scale, across real-time communication, vision, and language workloads. You will optimize kernel-level performance, work on inference frameworks, and shape delivery pipelines with research and platform teams.

You will collaborate with research to translate model changes into efficient implementations, profile CUDA kernels, and ensure latency SLOs and cost goals are met.

Qualifications

• Bachelor's or Master's degree in Computer Science, Electrical Engineering, or related field.
• 5+ years of software engineering experience, with focus on inference systems or ML infra.
• Hands-on experience with at least one major inference framework.

Responsibilities

• Design and build high-performance inference serving systems for large-scale transformer and multimodal models (100B+ and MoE).
• Implement and tune inference optimisations: speculative decoding, continuous batching, KV cache management, prefill/decode disaggregation, and quantisation (INT4/INT8/FP8).
• Contribute to and customise inference frameworks (vLLM, TensorRT-LLM, SGLang, or equivalent) for production needs.
• Write and profile CUDA kernels and custom ops where framework-level optimisation is insufficient.
• Own end-to-end deployment: packaging, serving API design, latency SLO monitoring, and incident response.
• Partner with research to translate model changes into inference-efficient implementations.
• Drive technical design and set the bar for inference engineering practices.

Skills

5+ years software engineering

inference systems

CUDA kernel development

Python and C++

production ML inference

speculative decoding

continuous batching

KV cache design

quantisation pipelines

Education

Bachelor's or Master's in CS/EE

Tools

vLLM

TensorRT-LLM

SGLang

ONNX Runtime

Job description

Zoom is seeking an experienced software engineer to design and own inference systems powering its AI models at production scale, across real-time communication, vision, and language workloads. You will optimize kernel-level performance, work on inference frameworks, and shape delivery pipelines with research and platform teams.

You will collaborate with research to translate model changes into efficient implementations, profile CUDA kernels, and ensure latency SLOs and cost goals are met.