Compiler Engineer

Hanzo; United States; Remote; USD 150,000 - 210,000; Full time

Benefits offered by this job

Remote-first

High autonomy

Opportunities to publish

Job summary

Hanzo is seeking an experienced compiler engineer to develop technology that speeds AI model inference across cloud and edge environments. You will optimize computation graphs, lower neural network representations, and build hardware-specific execution paths for GPUs, TPUs, and accelerators.

Responsibilities include creating optimizing compiler components and graph transformations, generating efficient code and kernel calls, and improving memory management with techniques like KV caching and

Qualifications

• Strong C++ and/or Rust skills with deep compiler architecture knowledge.
• Familiarity with MLIR/LLVM and ML graph compilation tools (TVM, XLA, TensorRT, TorchDynamo).
• Understanding of GPU architecture, tensor cores, and warp scheduling.

Responsibilities

• Create optimizing compiler components and graph transformation passes for large language model inference.
• Use compiler IRs to generate efficient machine code and kernel calls.
• Improve memory management and execution techniques, including caching and attention optimizations.
• Benchmark and profile performance with focus on throughput and cost per generated token.

Skills

C++

Rust

Compiler architecture

GPU architecture

Performance measurement

Tools

MLIR

LLVM

TVM

XLA

TensorRT

TorchDynamo

Job description

Role overview

Develop compiler technology that makes AI model inference faster and more efficient across cloud and edge environments. You’ll optimize computation graphs, lower neural network representations, and build hardware-specific execution paths for GPUs, TPUs, and newer accelerator types.

Responsibilities

• Create optimizing compiler components and graph transformation passes for large language model inference.
• Use compiler intermediate representations to generate efficient machine code and kernel calls.
• Improve memory management and execution techniques, including KV caching, speculative decoding, and paged attention.
• Benchmark and profile performance, with attention to throughput and cost per generated token.

Requirements

• Strong C++ and/or Rust skills and a deep understanding of compiler architecture, including frontends, intermediate representations, optimization passes, and backends.
• Familiarity with MLIR, LLVM, or machine learning graph compilation tools such as TVM, XLA, TensorRT, or TorchDynamo.
• Understanding of GPU architecture, including tensor cores, shared memory, compute capabilities, and warp scheduling.
• Careful approach to performance measurement, micro-benchmarks, and regression testing.

Nice to have

• Experience writing CUDA or Metal kernels for attention or fused operations.
• Contributions to open-source compiler or runtime projects.

Benefits and work setup

• Remote-first work with a globally distributed team.
• High autonomy and ownership, with opportunities to publish work that developers can inspect and build on.