Senior AI Inference Platform Engineer

Aleroy; United Arab Emirates; Hybrid; AED 360,000 - 620,000; Full time

Benefits offered by this job

Real ownership of backend

Performance-based incentives

24 days annual leave

Health insurance

Modern office in Yas Creative Hub

Job summary

Aleroy seeks a senior AI-inference engineer to own our serving stack across inference, metrics, and testing. You will lead performance improvements for large-scale LLM workloads, optimize GPU utilization, and drive MoE serving and quantization strategies.

The role combines hands-on engineering with technical leadership in distributed AI infrastructure, benchmarking, and production-grade improvements. Based in UAE with a strong focus on performance and reliability.

Qualifications

• Hands-on experience building or operating LLM inference-serving systems in production.
• Deep understanding of inference stacks (vLLM, SGLang, TensorRT-LLM) or similar.
• Knowledge of paged attention, continuous batching, and disaggregated serving.
• Experience with CUDA or Triton for GPU performance.
• Experience serving Mixture-of-Experts (MoE) models and quantization.
• Strong benchmarking, latency/throughput measurement, and reproducibility.
• Distributed systems expertise in routing and scheduling across hardware.
• Technical leadership in inference/ML infra projects.
• Ability to set direction while remaining hands-on.
• Preferred: experience with speculative decoding and open-source contributions.

Responsibilities

• Own and improve the INF serving stack, including batching and KV-cache management.
• Improve throughput, latency, memory efficiency, and cost while preserving model quality.
• Identify GPU bottlenecks and optimize using CUDA/Triton.
• Drive efficient MoE serving with expert parallelism and load balancing.
• Implement and evaluate quantization and other optimisations with accuracy checks.
• Own reproducible benchmarking across performance, quality, and cost.
• Build instrumentation to diagnose system behavior and validate improvements.
• Make technical decisions on routing, scheduling, and workload distribution.
• Define the AI serving roadmap and prioritize infra improvements.
• Set and enforce engineering standards for performance, benchmarking, and production readiness.
• Mentor engineers while contributing to implementation and problem solving.

Skills

LLM inference-serving

Performance tuning

Distributed systems

Engineering leadership

Benchmarking

CUDA/Triton

MoE serving

Quantization

Speculative decoding

KV-cache management

Model optimisation

Performance instrumentation

Distributed routing/scheduling

English (Advanced, C1)

Tools

vLLM

SGLang

TensorRT-LLM

CUDA

Triton

Job description

Aleroy seeks a senior AI-inference engineer to own our serving stack across inference, metrics, and testing. You will lead performance improvements for large-scale LLM workloads, optimize GPU utilization, and drive MoE serving and quantization strategies.

The role combines hands-on engineering with technical leadership in distributed AI infrastructure, benchmarking, and production-grade improvements. Based in UAE with a strong focus on performance and reliability.