Runtime Engineer

Oho Group; San Francisco (CA); On-site; USD 180,000 - 240,000; Full time

Job summary

Oho Group in San Francisco is seeking a Runtime Engineer to build the execution layer of their compiler platform for AI workloads. You will translate optimized compiler output into efficient, scalable code across modern hardware.

This role focuses on parallel execution, kernel scheduling, runtime architecture, and performance analysis, working close to hardware and with compiler engineers to turn ideas into real-world gains.

Qualifications

• 4+ years working on compiler or runtime systems.
• Strong modern C++.
• Deep understanding of asynchronous and concurrent programming.
• Familiarity with memory hierarchies and vector/scalar execution.
• OS kernel or hypervisor development experience.
• GPU programming with CUDA/ROCm, HPC, PyTorch, JAX or Triton a plus.

Responsibilities

• Design and develop a multi-target runtime for AI workloads.
• Build parallelisation, partitioning and kernel-scheduling capabilities.
• Prototype and evaluate new runtime approaches using performance data.
• Benchmark compiler output on target hardware and diagnose bottlenecks.
• Develop profiling and analysis tools that guide runtime and compiler improvements.
• Work with product and compiler teams to evolve the platform around real ML-engineering needs.

Skills

Compiler/runtime experience

Modern C++

Async/Concurrent

Memory Hierarchy

Kernel/Hypervisor

GPU CUDA/ROCm / Triton

PyTorch/JAX/Triton experience

Job description

We are working with an ambitious AI infrastructure company building a new software stack that enables machine-learning workloads to run efficiently across diverse hardware targets.

They are looking for a Runtime Engineer to build the execution layer at the heart of their compiler platform. You will take optimised compiler output and make it run efficiently, correctly and at scale across modern AI hardware.

This is a deeply systems-focused role spanning parallel execution, kernel scheduling, runtime architecture and performance analysis. You will work close to hardware while collaborating directly with compiler engineers to turn optimisation ideas into real-world performance.

What you’ll do

• Design and develop a multi-target runtime for AI workloads
• Build parallelisation, partitioning and kernel-scheduling capabilities
• Prototype and evaluate new runtime approaches using performance data
• Benchmark compiler output on target hardware and diagnose bottlenecks
• Develop profiling and analysis tools that guide runtime and compiler improvements
• Work with product and compiler teams to evolve the platform around real ML-engineering needs

What we’re looking for

• 4+ years working on compiler or runtime systems
• Strong modern C++
• Deep understanding of asynchronous and concurrent programming
• Familiarity with memory hierarchies and vector/scalar execution
• OS kernel or hypervisor development experience
• GPU programming, CUDA/ROCm, HPC, large clusters, PyTorch, JAX or Triton experience would be valuable

You will build the runtime that determines whether next-generation AI infrastructure merely works or genuinely performs. It is a rare opportunity to work across hardware intrinsics, compiler output and distributed execution, with real ownership over a foundational systems product.