Senior AI Inference Engineer — High-Performance Serving

Aleroy; Abu Dhabi; On-site; AED 360,000 - 600,000; Full time

Benefits offered by this job

Health insurance

24 days annual leave

Performance-based incentives

Modern office in Yas Creative Hub

Job summary

Aleroy is building a unified AI inference platform in Abu Dhabi, offering a powerful back-end for high‑performance AI workloads. This senior role leads inference serving, performance engineering, and MoE deployment across diverse hardware, guiding a small, senior team in a hands-on, architecture‑level capacity.

You will own the serving stack, optimize GPU performance, and implement quantization while ensuring measurement rigor and reproducibility.

Qualifications

• Hands-on experience building, optimising, and operating LLM inference-serving systems in production.
• Deep understanding of inference stacks such as vLLM, SGLang, TensorRT-LLM, or similar.
• Strong knowledge of paged attention, continuous batching, disaggregated serving, KV-cache management, and modern inference architectures.
• Proficiency with CUDA/Triton for GPU optimisation.
• Experience with MoE models, expert parallelism, and distributed execution.
• Quantization and model optimisation with careful accuracy measurement.
• Benchmarking at scale with latency percentiles and SLO-aligned metrics.
• Experience in distributed systems routing, scheduling and workload distribution.
• Leadership in ML infra initiatives and hands-on implementation.

Responsibilities

• Own and improve the FAR Labs inference serving stack, including prefill/disaggregation and advanced batching.
• Improve throughput, latency, memory efficiency, and cost with model-quality standards.
• Identify and remove GPU bottlenecks using CUDA/Triton and profiling.
• Drive efficient MoE serving with expert parallelism and load balancing.
• Implement and evaluate quantization and other optimisations with accuracy checks.
• Lead reproducible benchmarking across performance, quality and cost.
• Develop instrumentation to understand system behaviour and bottlenecks.
• Make routing, scheduling, and caching decisions across diverse hardware.

Skills

LLM inference serving

CUDA / Triton

MoE models

Performance benchmarking

Distributed systems

Model quantization

Speculative decoding

Open-source contributions

English (Advanced)

Tools

vLLM

SGLang

TensorRT-LLM

Job description

Aleroy is building a unified AI inference platform in Abu Dhabi, offering a powerful back-end for high‑performance AI workloads. This senior role leads inference serving, performance engineering, and MoE deployment across diverse hardware, guiding a small, senior team in a hands-on, architecture‑level capacity.

You will own the serving stack, optimize GPU performance, and implement quantization while ensuring measurement rigor and reproducibility.