Inference Performance Engineer for Visual AI

AI Chopping Block, Inc.; San Francisco (CA); On-site; USD 150,000 - 230,000; Full time

Benefits offered by this job

Equity

401k

Healthcare

Lunch & snacks

Job summary

Hedra in San Francisco is seeking an Inference Optimization Engineer to join our research-and-systems team. You will push state-of-the-art visual models to run faster at inference time across single and multi-GPU environments.

Role spans algorithmic and systems optimizations, including quantization and advanced memory techniques, with close collaboration with researchers and engineers to deploy scalable solutions.

Qualifications

• Deep technical ability in efficient ML inference, ML systems, GPU computing, or adjacent research.
• Strong understanding of how modern deep learning models execute on hardware.
• Experience profiling ML workloads, identifying bottlenecks, and driving improvements.
• Strong programming fundamentals in Python, C++, or another systems language.
• Experience with PyTorch, CUDA, Triton, TensorRT, vLLM, or comparable technologies.

Responsibilities

• Work with researchers and engineers to make new visual models fast and efficient at inference time.
• Profile architectures and workloads to identify bottlenecks in compute, memory, and communication.
• Develop approaches to reduce latency, increase throughput, and improve memory efficiency.
• Explore algorithmic optimizations including quantization, sparsity, and memory movement.
• Build or optimize GPU kernels using CUDA or Triton when needed.
• Optimize execution across single- and multi-GPU, and multi-node setups.
• Reason about architecture-hardware interactions to unlock performance gains.
• Develop benchmarking and profiling infrastructure to evaluate ideas.
• Evaluate new runtimes, compilers, and accelerator hardware.

Skills

Python

C++

GPU computing

Profiling ML workloads

PyTorch

CUDA

Triton

TensorRT

vLLM

SGLang

Tools

Nsight Systems

Nsight Compute

CUTLASS

GPU kernel development

Job description

Hedra in San Francisco is seeking an Inference Optimization Engineer to join our research-and-systems team. You will push state-of-the-art visual models to run faster at inference time across single and multi-GPU environments.

Role spans algorithmic and systems optimizations, including quantization and advanced memory techniques, with close collaboration with researchers and engineers to deploy scalable solutions.