Senior AI Inference Engineer, Enterprise Platform & GPUs

Uniphore; Palo Alto (CA); On-site; USD 159,000 - 199,000; Full time

Benefits offered by this job

Annual incentive

Stock options

Medical, dental, vision

401(k) with match

Paid time off

Birthday leave

Job summary

Uniphore is seeking a Senior or Staff Inference Engineer to own the inference layer for an enterprise AI platform. You will combine deep model inference expertise with systems engineering to deliver context-aware runtime serving finetuned SLMs across cloud, on-premises, and GPU platforms.

You will build and optimize KV-cache, speculative decoding, and multi-GPU execution, while managing model and adapter lifecycles and ensuring reliable production deployments with strong observability.

Qualifications

• 5+ years engineering, including 3+ years hands-on with GPU systems, distributed inference, or ML serving.
• Production experience with vLLM, SGLang, or TensorRT-LLM.
• Strong Python and C++ (C++17/20); ability to read and debug CUDA C++ or HIP kernels.
• Hands-on GPU profiling and performance optimization.

Responsibilities

• Build context-aware runtime: integrate context selection, compression, conversation compaction, and persistent task state with model execution.
• KV-cache and prompt-prefix-cache management, cache-aware routing, retention, eviction, and offloading.
• Inference performance: implement and tune speculative decoding, attention kernels, continuous batching, quantization, and multi-GPU execution.
• Adapter lifecycle: model/adapter lifecycle management, multi-adapter serving, zero-downtime hot swapping, and resource scheduling.
• Production reliability: observability, tenant isolation, overload protection, failure recovery, safe rollout and rollback.
• Cross-environment deployment: portable runtime integrations across NVIDIA and AMD GPUs, cloud and on-prem, single or multi-GPU.

Skills

Python

C++

GPU systems

Distributed inference

ML serving

Performance profiling

Tools

CUDA

HIP

Triton

CUTLASS

Job description

Uniphore is seeking a Senior or Staff Inference Engineer to own the inference layer for an enterprise AI platform. You will combine deep model inference expertise with systems engineering to deliver context-aware runtime serving finetuned SLMs across cloud, on-premises, and GPU platforms.

You will build and optimize KV-cache, speculative decoding, and multi-GPU execution, while managing model and adapter lifecycles and ensuring reliable production deployments with strong observability.