Lead Inference Engineer — AI Serving & GPU Optimization

Aleroy; United States; Remote; USD 140,000 - 190,000; Full time

Job summary

Aleroy seeks a senior engineer to own and enhance the AI inference serving stack in the United States. You will drive performance through CUDA and Triton, implement efficient batching and cache strategies, and mentor engineers while staying hands-on with critical implementations.

You will help shape the roadmap for serving and measurement, establish engineering standards, and push for first-class reliability and scalability across inference workloads.

Qualifications

• Own and improve the AI inference serving stack (prefill/decode disaggregation, batching, KV-cache, speculative decoding).
• Identify and eliminate GPU bottlenecks using CUDA, Triton, profiling, and kernel optimization.
• Set the serving/measurement roadmap, establish engineering standards, and mentor engineers while remaining hands-on.

Skills

GPU performance engineering

CUDA

Triton

Profiling

Kernel optimization

Mentorship

Hands-on implementation

Tools

Triton Inference Server

CUDA Profiling

Job description

Aleroy seeks a senior engineer to own and enhance the AI inference serving stack in the United States. You will drive performance through CUDA and Triton, implement efficient batching and cache strategies, and mentor engineers while staying hands-on with critical implementations.

You will help shape the roadmap for serving and measurement, establish engineering standards, and push for first-class reliability and scalability across inference workloads.