Senior ML Engineer - Remote, High-Performance LLM Inference

Red Hat; United States; Hybrid; USD 186,000 - 287,000; Full time

Job summary

Red Hat is seeking a senior engineer to design and optimize high-performance Python and C++ code for vLLM-based inference, GPU kernels, and numerical components in production-scale LLM serving environments.

Responsibilities include benchmarking on NVIDIA GPUs, tuning memory and scheduling, and integrating cutting-edge serving techniques. This role emphasizes code quality, peer reviews, and upstream open-source collaboration.

Qualifications

• 3 years of high-performance Python and Modern C++ development.
• 3 years of GPU profiling on NVIDIA GPUs using profiling tools.
• 2 years of open-source contributions to PyTorch or vLLM; CUDA/Triton kernel optimization.

Responsibilities

• Architect, develop, and optimize advanced LLM inference algorithms for production-scale serving.
• Profile, benchmark, and model performance on NVIDIA GPUs; reduce latency and cost.
• Own end-to-end optimization of vLLM-based inference pipelines across multi-GPU setups.
• Prototype and integrate state-of-the-art LLM serving techniques (quantization, batching).
• Set engineering standards, conduct peer reviews, and guide testing/benchmarking.

Skills

Python

C++

CUDA

Triton

GPU profiling

Education

Master's degree in Computer Science / Computer Engineering

Bachelor's degree in Computer Science / Computer Engineering

Tools

Nsight Systems

Nsight Compute

CUPTI

PyTorch Profiler

CUDA

Triton

CUTLASS

Job description

Red Hat is seeking a senior engineer to design and optimize high-performance Python and C++ code for vLLM-based inference, GPU kernels, and numerical components in production-scale LLM serving environments.

Responsibilities include benchmarking on NVIDIA GPUs, tuning memory and scheduling, and integrating cutting-edge serving techniques. This role emphasizes code quality, peer reviews, and upstream open-source collaboration.