Remote Engineering Lead, Inference Optimization

Shields Group Search; United States; On-site; USD 270,000 - 330,000; Full time

Benefits offered by this job

Equity

Crypto token compensation

Job summary

Shields Group Search is partnering with a privacy‑focused consumer AI company to hire an Engineering Lead for Inference Optimization. This role blends hands‑on work with people leadership to shape the company’s inference stack and scale GPU performance.

You’ll drive latency improvements, build benchmarks, and guide a small team of engineers. Strong GPU, Python (plus Rust/Go) and LLM inference experience are expected, with equity and crypto token compensation alongside a base salary of

Qualifications

• 8+ years in performance optimization or HPC, with deep GPU architecture knowledge.
• 5+ years of experience leading engineering teams.
• Proficiency in Python, Rust, or Go.
• Hands-on experience with production LLM inference engines (e.g. vLLM, SGLang).
• Experience with inference optimization techniques and CUDA/Triton tooling.

Responsibilities

• Own the company’s technical strategy for inference performance.
• Recruit and lead the Inference Optimization Team.
• Optimize GPU infrastructure across architectures (e.g. H200s, B300s).
• Improve latency, throughput and cost per token for LLM inference.
• Build benchmarking harnesses to compare engines and quantization.
• Work with the routing system to optimize load balancing.
• Evaluate new techniques and hardware for viability.

Skills

Performance optimization

GPU architecture

Parallel programming

Team leadership

Python

Rust

Go

LLM inference engines

Latency optimization

Profiling tools

Tools

Nsight Systems

Nsight Compute

PyTorch Profiler

Torch.compile

CUDA

Triton kernels

vLLM

SGLang

Job description

Shields Group Search is partnering with a privacy‑focused consumer AI company to hire an Engineering Lead for Inference Optimization. This role blends hands‑on work with people leadership to shape the company’s inference stack and scale GPU performance.

You’ll drive latency improvements, build benchmarks, and guide a small team of engineers. Strong GPU, Python (plus Rust/Go) and LLM inference experience are expected, with equity and crypto token compensation alongside a base salary of