ML Inference Systems Engineer - Accelerator Performance

Arago; Paris; On-site; EUR 120,000 - 180,000; Full time

Benefits offered by this job

Stock options

Healthcare coverage

Pension contributions

Professional development

25 days PTO

Job summary

Arago in Paris is building a cutting-edge AI accelerator and seeks a senior engineer to optimize model execution and inference serving. You will work across kernels, runtimes, and multi-device distribution, shaping the software stack around Arago's hardware.

Ideal candidates have deep experience in high-performance ML inference, CUDA/Triton/ROCm, C++ and Python, and a proven track record with modern inference-serving systems. Fluency in English required; French is a plus.

Qualifications

• Extensive experience in high-performance ML inference and accelerator programming.
• Deep understanding of computer architecture, memory hierarchies and performance bottlenecks.
• Experience developing custom kernels with CUDA, Triton, ROCm/HIP or similar.
• Familiar with operator fusion, tiling, scheduling and data movement optimization.
• Experience with distributed model execution and parallelism.
• Hands-on experience with modern inference-serving systems (e.g., vLLM, TensorRT-LLM) including KV-cache management.
• Strong C++ and Python skills, with MLIR exposure a plus.
• Proficient English.

Responsibilities

• Analyze modern AI workloads and identify bottlenecks at kernel, runtime, memory, and system levels.
• Develop and optimize kernels, fused operators, and execution strategies to maximize device utilization.
• Design efficient mappings of models and operators across multiple Arago devices, including communication and synchronization strategies.
• Develop inference-serving techniques such as continuous batching, paged KV caches, and prefill/decode strategies.
• Build profiling, benchmarking, and performance-analysis infrastructure spanning kernels, full models, and serving workloads.
• Collaborate with hardware, compiler, and runtime teams to influence future software abstractions.

Skills

High-performance ML inference

GPU/accelerator programming

Distributed model execution

C++ and Python

MLIR familiarity

Tools

CUDA

Triton

ROCm/HIP

MLIR dialects

Job description

Arago in Paris is building a cutting-edge AI accelerator and seeks a senior engineer to optimize model execution and inference serving. You will work across kernels, runtimes, and multi-device distribution, shaping the software stack around Arago's hardware.

Ideal candidates have deep experience in high-performance ML inference, CUDA/Triton/ROCm, C++ and Python, and a proven track record with modern inference-serving systems. Fluency in English required; French is a plus.