AI Runtime Engineer: Multi-Target Performance & Scheduling

Oho Group; San Francisco (CA); On-site; USD 180,000 - 240,000; Full time

Job summary

Oho Group in San Francisco is seeking a Runtime Engineer to build the execution layer of their compiler platform for AI workloads. You will translate optimized compiler output into efficient, scalable code across modern hardware.

This role focuses on parallel execution, kernel scheduling, runtime architecture, and performance analysis, working close to hardware and with compiler engineers to turn ideas into real-world gains.

Qualifications

• 4+ years working on compiler or runtime systems.
• Strong modern C++.
• Deep understanding of asynchronous and concurrent programming.
• Familiarity with memory hierarchies and vector/scalar execution.
• OS kernel or hypervisor development experience.
• GPU programming with CUDA/ROCm, HPC, PyTorch, JAX or Triton a plus.

Responsibilities

• Design and develop a multi-target runtime for AI workloads.
• Build parallelisation, partitioning and kernel-scheduling capabilities.
• Prototype and evaluate new runtime approaches using performance data.
• Benchmark compiler output on target hardware and diagnose bottlenecks.
• Develop profiling and analysis tools that guide runtime and compiler improvements.
• Work with product and compiler teams to evolve the platform around real ML-engineering needs.

Skills

Compiler/runtime experience

Modern C++

Async/Concurrent

Memory Hierarchy

Kernel/Hypervisor

GPU CUDA/ROCm / Triton

PyTorch/JAX/Triton experience

Job description

Oho Group in San Francisco is seeking a Runtime Engineer to build the execution layer of their compiler platform for AI workloads. You will translate optimized compiler output into efficient, scalable code across modern hardware.

This role focuses on parallel execution, kernel scheduling, runtime architecture, and performance analysis, working close to hardware and with compiler engineers to turn ideas into real-world gains.