LLM Inference Engineer — High-Performance AI Serving

F5; San Jose (CA); On-site; USD 140,000 - 210,000; Full time

Job summary

F5 is seeking an AI Inference Engineer to bridge high-performance model development with optimized deployment. You will optimize Large Language Models for inference across GPU-rich data centers and edge devices, focusing on throughput, latency, and accuracy.

You will work on hardware acceleration, scalable infrastructure, and performance monitoring to ensure enterprise-grade reliability and efficient AI capabilities.

Qualifications

• Experience building high-performance AI workflows and inference systems.
• Strong hands-on with inference tooling and model optimization.
• Experience with Docker, Kubernetes and cloud platforms (AWS, GCP, Azure).

Responsibilities

• Build and maintain high-performance AI inference engines at scale.
• Profile and optimize models for NVIDIA GPUs, Apple Silicon, and AI accelerators.
• Design auto-scaling architectures for online and batch inference using Kubernetes.
• Establish observability for TTFT, tokens/second, and memory bandwidth against SLAs.

Skills

Python

C++

Rust

Golang

Tools

vLLM

TensorRT

Llama.cpp

Ollama

Docker

Kubernetes

AWS

GCP

Azure

Job description

F5 is seeking an AI Inference Engineer to bridge high-performance model development with optimized deployment. You will optimize Large Language Models for inference across GPU-rich data centers and edge devices, focusing on throughput, latency, and accuracy.

You will work on hardware acceleration, scalable infrastructure, and performance monitoring to ensure enterprise-grade reliability and efficient AI capabilities.