ML Inference Engineer | Accelerate AI on Custom Accelerator

Arago; Paris; On-site; EUR 90,000 - 130,000; Full time

Benefits offered by this job

Stock options

Healthcare coverage

Pension contributions

Professional development

25 days PTO

Job summary

Arago is seeking an experienced engineer to optimize AI model execution on its custom accelerator. You will work across kernels, model execution, multi-device distribution, and inference serving, shaping the software stack around the hardware capabilities.

You will contribute to high-performance ML inference, kernel optimization, and distributed execution while collaborating with hardware, compiler, and runtime teams to drive performance improvements.

Qualifications

• Strong experience in high-performance ML inference and accelerator programming.
• Deep understanding of computer architecture, memory hierarchies, and parallelism.
• Experience developing and optimizing custom kernels using low-level environments (CUDA, Triton, ROCm/HIP).
• Experience with operator fusion, tiling, scheduling, data movement optimization, and graph execution.

Responsibilities

• Analyze modern AI workloads to identify kernel-, runtime-, memory-, and system-level bottlenecks on Arago's accelerator.
• Develop and optimize custom kernels and fused operators for maximum device utilization.
• Design efficient mappings of models across multiple Arago devices and manage communication/synchronization.
• Develop inference-serving techniques such as continuous batching and KV caches.
• Build profiling, benchmarking, and performance-analysis infrastructure spanning kernels and full models.

Skills

ML inference

GPU programming

Computer architecture

CUDA/Triton/ROCm

Kernels optimization

Distributed execution

Inference serving

C++

Python

MLIR

English proficiency

Tools

CUDA

TensorRT

ROCm

MLIR

Job description

Arago is seeking an experienced engineer to optimize AI model execution on its custom accelerator. You will work across kernels, model execution, multi-device distribution, and inference serving, shaping the software stack around the hardware capabilities.

You will contribute to high-performance ML inference, kernel optimization, and distributed execution while collaborating with hardware, compiler, and runtime teams to drive performance improvements.