Staff Inference Engineer - Enterprise AI Runtime on GPUs

Uniphore; Palo Alto (CA); On-site; USD 180,000 - 260,000; Full time

Job summary

Uniphore is building an enterprise AI platform where agents operate across voice, video, text, and structured data in real time. We seek a Senior or Staff Inference Engineer to own the inference layer and deliver a context-aware runtime serving finetuned SLMs across cloud, on-premises, and multiple GPU platforms.

This is a hands-on IC role collaborating with research, model engineering, knowledge systems, and platform teams to push performance, reliability, and portability across NVIDIA and AMD

Qualifications

• 5+ years engineering experience, including 3+ years with GPU systems or ML serving.
• Experience with production deployments of GPU-accelerated inference.
• Proficient in Python and C++, with ability to read CUDA/C++ kernels.

Responsibilities

• Own the inference layer across cloud, on-premises, and GPU platforms.
• Develop context-aware runtime, KV-cache, and prompt-prefix caching.
• Implement and optimize speculative decoding, attention kernels, and multi-GPU parallelism.
• Ensure production reliability with observability and safe rollout.

Skills

GPU systems

distributed inference

ML serving

Python

C++

CUDA

Tools

vLLM

SGLang

TensorRT-LLM

CUDA C++

HIP kernels

Job description

Uniphore is building an enterprise AI platform where agents operate across voice, video, text, and structured data in real time. We seek a Senior or Staff Inference Engineer to own the inference layer and deliver a context-aware runtime serving finetuned SLMs across cloud, on-premises, and multiple GPU platforms.

This is a hands-on IC role collaborating with research, model engineering, knowledge systems, and platform teams to push performance, reliability, and portability across NVIDIA and AMD