Paris-Based Compiler Engineer for LLM Inference

Kog; Paris; On-site; EUR 90,000 - 150,000; Full time

Job summary

Kog is building a co-designed inference stack to optimize LLM execution on datacenter GPUs. You will contribute to AGCO, designing and evaluating compiler and IR components, optimization passes, and verification workflows, with a focus on generating changes, compiling them, and measuring results on real hardware.

You will work across compilers, GPU systems, and LLM inference, taking ownership of technical decisions while collaborating with teammates across the full stack to push performance and

Qualifications

• We look for engineers with deep technical expertise and original work in at least one area relevant to AGCO.

Responsibilities

• Work on AGCO to explore optimizations for LLM execution.
• Build and verify compiler/IR design for LLM computations.
• Develop optimization passes, lowering, and code generation.
• Implement and evaluate search methods for different execution strategies.
• Perform GPU profiling, memory optimization, and performance tuning.
• Ensure correctness checks for generated changes and LLM operators.

Skills

Compiler engineering

GPU programming

LLM inference

Performance optimization

Formal verification

Code optimization systems

Attention MoE

SAT/SMT

Tools

CUDA

HIP

Metal

Vulkan

Job description

ABOUT KOG

Kog builds a co-designed inference stack for real-time AI agents on standard NVIDIA and AMD datacenter GPUs, spanning model architecture, inference engine, compilers, and low-level GPU kernels.

We developed Laneformer 2B and Delayed Tensor Parallelism (DTP), a Transformer architecture designed to overlap communication with computation.

Our inference stack generates 3,500 tokens/s per request on 8 AMD MI300X GPUs and 2,100 tokens/s per request on 8 NVIDIA H200 GPUs, in FP16 at batch size 1, without quantization or speculative decoding.

Our next challenge is making model optimization faster and more scalable. New models are released constantly, and spending months adapting each one to a specific GPU setup is not sustainable.

This is why we're developing AGCO, our agentic compiler initiative. The goal is to build on our existing optimization techniques to dramatically reduce the time needed to port and optimize models across different GPU architectures, progressively automating parts of that process.

We're a team of 10 people, including three PhD holders and one PhD candidate.

Try our technology at playground.kog.ai and read more on the Kog Labs blog.

WHAT YOU WILL WORK ON

You will help design and build AGCO, contributing to:

• Compiler and IR design, optimization passes, lowering, and code generation.

• GPU execution, kernel optimization, and performance profiling.

• Systems that explore different implementations and execution strategies.

• Verification, correctness checks, and benchmarking on real hardware.

• Optimization loops that use measured results to guide the next improvements.

We are still exploring the system's architecture. One direction combines an IR, a verifier, a compiler, and a search optimizer. You will help turn these ideas into working prototypes and determine what scales.

WHAT WE LOOK FOR

We look for engineers with strong technical expertise and original work in areas such as:

• Compiler engineering, including LLVM, MLIR, or similar technologies.

• GPU programming with CUDA, HIP, or comparable low-level tools.

• GPU performance optimization, memory, synchronization, and profiling.

• LLM inference engines, parallelism, and communication.

• Automated code generation, verification, or optimization systems.

You do not need expertise in every area. What matters is your technical depth, the systems you have personally built, and your ability to explain your design decisions and results.

We review existing technical work during the process, including code, upstream contributions, papers, theses, or technical projects.

WHAT WE OFFER

• Significant ownership over a new part of Kog's technology.

• Close collaboration with engineers working across the full inference stack.

• Direct access to NVIDIA and AMD datacenter GPUs.

• The opportunity to help build systems that change how quickly new AI models can be optimized and deployed.

Location: Paris, France. Candidates based elsewhere must be willing to relocate to Paris.