Senior ML Systems Engineer - Remote, GPU Inference

Red Hat, LLC; Boston (MA); On-site; USD 186,000 - 287,000; Full time

Job summary

Red Hat, LLC in Boston seeks a senior engineer to lead the design and implementation of high-performance Python, C++, CUDA, and Triton code for vLLM-based inference systems in production-scale environments. You will drive performance profiling and optimization on NVIDIA GPUs, expanding kernel and memory optimizations while advancing end-to-end inference pipelines across multi-GPU deployments.

This role requires a strong track record in open-source contributions and advanced GPU computing

Qualifications

• Mastery of high‑performance Python and modern C++ for large‑scale inference.
• Experience profiling and optimizing GPU workloads with Nsight or equivalent.
• Familiarity with GPU memory architectures and deep learning optimizations.

Responsibilities

• Architect, develop, and optimize LLM inference algorithms for vLLM-based systems.
• Profile, benchmark, and model latency, throughput, and cost on NVIDIA GPUs.
• Own end-to-end optimization of inference pipelines across multi‑GPU deployments.
• Prototype and integrate cutting‑edge serving techniques from research.
• Set engineering standards, perform peer reviews, and guide testing.

Skills

Python

C++

GPU optimization

Performance profiling

Open-source contributions

Education

Master's degree in Computer Science or related field

Bachelor's degree in Computer Science or related field

Tools

CUDA

Triton

CUTLASS

Nsight Systems

Nsight Compute

CUPTI

Job description

Red Hat, LLC in Boston seeks a senior engineer to lead the design and implementation of high-performance Python, C++, CUDA, and Triton code for vLLM-based inference systems in production-scale environments. You will drive performance profiling and optimization on NVIDIA GPUs, expanding kernel and memory optimizations while advancing end-to-end inference pipelines across multi-GPU deployments.

This role requires a strong track record in open-source contributions and advanced GPU computing