Low-Latency AI Inference Engineer

Relha LLC; San Jose (CA); Hybrid; USD 177,000 - 265,000; Full time

Job summary

F5 is seeking an AI Inference Engineer to bridge high-performance model development and optimized deployment environments. You will optimize Large Language Models for inference across GPUs, edge devices, and data centers, focusing on throughput, latency, and accuracy.

Responsibilities include building inference engines with vLLM, TensorRT, Llama.cpp, and Ollama, plus deploying scalable, low-latency solutions using Kubernetes and cloud platforms.

Qualifications

• Proficiency in Python, C++, Rust, or Golang for high-performance AI workflows.
• Hands-on experience with AI inference tools: vLLM, TensorRT, Llama.cpp, Ollama.
• Infrastructure expertise with Docker, Kubernetes, and cloud platforms (AWS, GCP, Azure).
• Strong understanding of GPU/AI hardware, profiling, and optimization for accelerators like NVIDIA GPUs and TPUs.

Responsibilities

• Build and maintain high-performance inference engines for scalable AI serving.
• Optimize deployment for low latency across GPU-rich data centers and edge devices.
• Profile and optimize models on NVIDIA GPUs (CUDA/TensorRT), Apple Silicon (CoreML), and TPUs/LPUs.
• Design auto-scaling architectures using Kubernetes for real-time and batch inference.
• Establish observability for TTFT, tokens per second, and memory bandwidth against SLAs.

Skills

Python

C++

Rust

Golang

Tools

vLLM

TensorRT

Llama.cpp

Ollama

Docker

Kubernetes

AWS

GCP

Azure

CUDA

Triton

Job description

F5 is seeking an AI Inference Engineer to bridge high-performance model development and optimized deployment environments. You will optimize Large Language Models for inference across GPUs, edge devices, and data centers, focusing on throughput, latency, and accuracy.

Responsibilities include building inference engines with vLLM, TensorRT, Llama.cpp, and Ollama, plus deploying scalable, low-latency solutions using Kubernetes and cloud platforms.