Tech Lead: LLM Inference & High-Performance Systems

wekatest

United States

Remote

USD 180,000 - 240,000

Full time

14 days+

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Benefits offered by this job

Medical insurance
Dental insurance
Vision insurance
Life insurance
Retirement benefits
Flexible time off
Sick leave
Statutory leave

Job summary

wekatest is seeking a lead software engineer to head a small team delivering high-throughput, low-latency inference infrastructure for large language models. You will design, build, and optimize systems across NVMe storage, GPU memory, and network paths, while mentoring three engineers and driving production reliability.

The role combines hands-on development with technical direction, requiring strong Python/C++ skills, CUDA expertise, and experience deploying GPU workloads on Kubernetes in a

Qualifications

  • 5+ years of professional software engineering experience including leading engineers and owning complex production systems.
  • Hands-on knowledge of LLM serving, including KV cache reuse, continuous batching, disaggregated prefill and decode, and tiered GPU memory.
  • Strong Python and C++ skills and a solid understanding of CUDA, GPU memory management, and high-performance I/O.

Responsibilities

  • Own technical decisions and delivery for inference infrastructure, including NVMe-backed token storage, GPU Direct Storage data paths, and model serving components.
  • Guide three developers through system design, implementation, and production delivery while maintaining standards for reliability and code quality.
  • Contribute directly to production systems using Python, C++, CUDA, and the surrounding infrastructure stack.
  • Address challenges such as separating prompt processing from token generation, persistent KV caching beyond GPU memory, RDMA transport, and tiered GPU memory.
  • Coach engineers through regular one-to-ones, career discussions, and sprint reviews.
  • Evaluate emerging inference frameworks and tools through benchmarking, helping determine when to adopt, build, or change direction.

Skills

Python
C++
CUDA
Kubernetes
Mentoring/leadership

Tools

GPUDirect Storage
RDMA
NVMe data paths
GPU storage
Bare-metal GPU clusters

Job description

wekatest is seeking a lead software engineer to head a small team delivering high-throughput, low-latency inference infrastructure for large language models. You will design, build, and optimize systems across NVMe storage, GPU memory, and network paths, while mentoring three engineers and driving production reliability.

The role combines hands-on development with technical direction, requiring strong Python/C++ skills, CUDA expertise, and experience deploying GPU workloads on Kubernetes in a

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Tech Lead - AI Inference
Tech Lead - AI Inference

wekatest • United States

Remote
USD 180,000 - 240,000
Medical insurance
Dental insurance
Vision insurance
+5
Senior Inference SW Engineer
Senior Inference SW Engineer

Hewlett Packard Enterprise • Fort Collins (CO)

On-site
USD 180,000 - 240,000
Health And Wellbeing Benefits
Personal And Professional Development
Lead, Scale GPU Inference Infrastructure (Remote)
Lead, Scale GPU Inference Infrastructure (Remote)

telnyx54 • United States

Remote
USD 180,000 - 320,000
Founding-seat influence on platform
Remote position
Open-source contribution & conference"
Senior Inference SW Engineer
Senior Inference SW Engineer

Hewlett Packard Enterprise • Durham (NC)

On-site
USD 140,000 - 180,000
Health And Wellbeing Benefits
Personal And Professional Development
ML Infra Tech Lead: High-Performance GPU & K8s
ML Infra Tech Lead: High-Performance GPU & K8s

Reducto • Santa Fe (NM)

On-site
USD 190,000 - 230,000
Unlimited PTO
Daily Lunch
Commuter Reimbursement
+3
LLM Inference Systems Engineer
LLM Inference Systems Engineer

Netpreme • Santa Clara (CA), Boston (MA)

On-site
USD 170,000 - 210,000
Competitive salary with bonus
Health, Dental, and Vision for you and
401(k) match with immediate vesting
+7
Senior Software Engineer, Inference
Senior Software Engineer, Inference

Hewlett Packard Enterprise • Durham (NC)

On-site
USD 150,000 - 210,000
Health benefits
Professional development programs
Flexible work arrangements
Principal Software Engineer, Inference
Principal Software Engineer, Inference

Hewlett Packard Enterprise • Fort Collins (CO)

Hybrid
USD 180,000 - 250,000
Health and wellbeing benefits
Professional development programs
Flexible work arrangements
LLM Inference Systems Lead
LLM Inference Systems Lead

WekaIO • United States

Remote
USD 220,000 - 320,000
Senior Inference Systems Engineer - Rust & GPUs
Senior Inference Systems Engineer - Rust & GPUs

Wintermeyer Ventures • Palo Alto (CA)

On-site
USD 230,000 - 350,000