Senior LLM Inference Architect — Hybrid/Remote

Red Hat, Inc.; Boston (MA); Hybrid; USD 186,000 - 287,000; Full time

Job summary

Red Hat, Inc. is seeking a Senior Machine Learning Engineer to lead the design and optimization of vLLM-based inference systems on hybrid Boston workloads. You will implement high-performance Python, C++, CUDA, and Triton code for production-scale LLM serving, focusing on latency, throughput, and cost efficiency.

The role requires deep GPU optimization expertise, strong collaboration with open-source communities, and the ability to guide performance-driven architecture and reviews across teams.

Qualifications

• 3 years of experience in high-performance Python and Modern C++ development.
• 3 years of profiling/benchmarking on NVIDIA GPUs using Nsight or similar tools.
• 2 years of experience with open-source PRs to PyTorch or vLLM; CUDA, Triton, or CUTLASS kernel optimization; LLM inference optimization; distributed computing; and leading technical design.

Responsibilities

• Architect, develop, and optimize advanced LLM inference algorithms including quantization, sparsification, batching, and memory management.
• Drive performance profiling and modeling on NVIDIA GPUs using Nsight Systems, Nsight Compute, CUPTI, PyTorch Profiler, CUDA, Triton, and CUTLASS.
• Own end-to-end optimization of vLLM-based inference pipelines, including kernel-level tuning and multi-GPU serving.
• Evaluate, prototype, and integrate state-of-the-art LLM serving techniques from research and open-source communities.
• Set engineering standards, conduct peer reviews, and guide testing and benchmarking strategies.

Skills

Python performance

Modern C++

Nsight profiling

GPU memory optimization

Open-source PRs

CUDA/Triton/CUTLASS

LLM inference

Distributed computing

Technical leadership

Education

Master's degree in CS/CE or related field

Bachelor's degree in CS/CE or related field

Job description

Red Hat, Inc. is seeking a Senior Machine Learning Engineer to lead the design and optimization of vLLM-based inference systems on hybrid Boston workloads. You will implement high-performance Python, C++, CUDA, and Triton code for production-scale LLM serving, focusing on latency, throughput, and cost efficiency.

The role requires deep GPU optimization expertise, strong collaboration with open-source communities, and the ability to guide performance-driven architecture and reviews across teams.