Senior GenAI Inference & GPU Performance Engineer

Advanced Micro Devices, Inc.; San Jose (CA); On-site; USD 220,000 - 320,000; Full time

Job summary

Advanced Micro Devices, Inc. is seeking a Principal GenAI Inference Optimization Engineer to advance GenAI inference performance on AMD GPUs. You’ll optimize latency, throughput, and cost for large-scale models across single-node and distributed deployments.

You will work across kernels, runtimes, and serving frameworks, collaborating with cross-functional teams to push the limits of GenAI deployment and efficiency.

Qualifications

• Strong understanding of GPU architecture and performance fundamentals.
• Experience with GenAI inference optimization techniques (e.g., quantization, KV-cache optimization, batching).
• Hands-on experience with inference/serving frameworks such as vLLM, SGLang, Triton, TensorRT-LLM, or similar.
• Experience with LLM or multimodal inference workloads.
• Familiarity with distributed systems and serving architectures.
• Experience with ML frameworks (PyTorch, JAX, or TensorFlow), especially for inference.
• Proficiency in Python and at least one systems language (C++/CUDA/HIP).
• Experience with profiling, debugging, and performance tuning tools.
• Ability to work collaboratively across teams and deliver impactful optimizations.

Responsibilities

• Optimize performance of GenAI inference workloads on AMD GPU platforms across single-node and distributed environments.
• Improve latency, throughput, and cost efficiency for LLM and multimodal model serving in production.
• Analyze and resolve bottlenecks across compute, memory, and communication.
• Contribute to cross-stack optimizations spanning kernels, runtimes, communication libraries, and inference/serving frameworks.
• Implement and evaluate inference optimization techniques such as batching, quantization, prefix caching, and speculative decoding.
• Support development and optimization of scalable serving systems, including request scheduling and resource utilization.
• Develop and use profiling, benchmarking, and performance analysis tools for inference workloads.
• Collaborate with hardware, compiler, and framework teams to improve overall system performance.
• Contribute to internal tools and open-source projects for inference optimization on AMD platforms.
• Document best practices and contribute to performance guidelines for GenAI deployment.

Skills

GenAI inference optimization

GPU architecture

LLM inference workloads

Distributed systems

ML frameworks

Python

C++/CUDA/HIP

Profiling & performance tuning

Cross-functional collaboration

Education

B.S., M.S. or Ph.D. in Computer Science/Engineering

Tools

vLLM

SGLang

Triton

TensorRT-LLM

PyTorch

JAX

TensorFlow

Job description

Advanced Micro Devices, Inc. is seeking a Principal GenAI Inference Optimization Engineer to advance GenAI inference performance on AMD GPUs. You’ll optimize latency, throughput, and cost for large-scale models across single-node and distributed deployments.

You will work across kernels, runtimes, and serving frameworks, collaborating with cross-functional teams to push the limits of GenAI deployment and efficiency.