AI Research Engineer: Distributed Training & MoE Models

European Tech Recruit; Paris; Sur place; EUR 90 000 - 130 000; Plein temps

Résumé du poste

A cutting-edge AI safety startup is seeking an AI Research Engineer in Paris to tackle complex training challenges for LLMs and VLMs. The role focuses on designing next-generation architectures, building multimodal training pipelines, and optimizing performance through advanced distributed training frameworks like Megatron and DeepSpeed. Candidates should possess a deep understanding of distributed training concepts and Triton kernels. This is a full-time position with a mid-senior level requirement, offering the opportunity to drive innovation in advanced AI systems.

Qualifications

• Deep understanding of tensor parallelism, pipeline parallelism, expert parallelism, ZeRO, and FSDP.
• Ability to write Triton kernels for custom operators or fundamental layers like attention and MLP.
• Experience working with multimodal systems like Llava-style and Flamingo-style.

Responsabilités

• Train LLMs and VLMs using advanced distributed training frameworks.
• Design and implement next-generation architectures, including Mixture-of-Experts (MoE).
• Build and maintain complex multimodal training pipelines for text, images, and audio data.
• Develop custom Triton kernels to optimize training and inference bottlenecks.
• Drive innovation exploring new model architectures and hyperparameter strategies.

Connaissances

Distributed training concepts

Triton kernels

Multimodal systems

Outils

Megatron

DeepSpeed

Description du poste

A cutting-edge AI safety startup is seeking an AI Research Engineer in Paris to tackle complex training challenges for LLMs and VLMs. The role focuses on designing next-generation architectures, building multimodal training pipelines, and optimizing performance through advanced distributed training frameworks like Megatron and DeepSpeed. Candidates should possess a deep understanding of distributed training concepts and Triton kernels. This is a full-time position with a mid-senior level requirement, offering the opportunity to drive innovation in advanced AI systems.