Compiler Runtime Engineer

Oho Group; San Francisco (CA); On-site; USD 150,000 - 210,000; Full time

Job summary

Oho Group is seeking a Runtime Engineer to turn optimized compiler output into reliable, high-performance execution for AI workloads. You will design a portable runtime, enabling parallel execution, scheduling, and profiling across diverse hardware.

You will collaborate with compiler and hardware engineers on runtime architecture, profiling, and performance analysis to push the platform forward for real ML workloads and deployment needs.

Qualifications

• 4+ years in compiler, runtime or low-level systems engineering.
• Strong modern C++ proficiency.
• Deep knowledge of concurrency and asynchronous execution.
• Understanding of memory hierarchies and vector/scalar compute.
• Experience with OS kernels, hypervisors or similarly low-level systems.

Responsibilities

• Design a portable runtime for AI workloads.
• Build workload partitioning, parallel execution and kernel scheduling capabilities.
• Prototype new runtime techniques and evaluate them against real hardware.
• Benchmark compiled workloads and diagnose performance bottlenecks.
• Develop profiling tools that inform compiler and runtime improvements.
• Help evolve the platform around real ML workloads and deployment requirements.

Skills

Modern C++

Concurrency

Memory hierarchy

Low-level systems

Tools

CUDA

ROCm

PyTorch

JAX

Triton

GPU programming

Job description

I’m working with an ambitious AI infrastructure company developing a compiler and runtime platform that enables machine-learning workloads to execute efficiently across different hardware architectures.

They are looking for a Runtime Engineer to build the systems layer responsible for turning optimized compiler output into reliable, high-performance execution. You will work closely with compiler and hardware engineers across runtime architecture, parallel execution, scheduling and performance analysis.

You will:

• Design a portable runtime for AI workloads
• Build workload partitioning, parallel execution and kernel-scheduling capabilities
• Prototype new runtime techniques and evaluate them against real hardware
• Benchmark compiled workloads and diagnose performance bottlenecks
• Develop profiling tools that inform compiler and runtime improvements
• Help evolve the platform around real ML workloads and deployment requirements

They are looking for:

• 4+ years in compiler, runtime or low-level systems engineering
• Strong modern C++
• Deep knowledge of concurrency and asynchronous execution
• Understanding of memory hierarchies and vector/scalar compute
• Experience with OS kernels, hypervisors or similarly low-level systems

Experience with CUDA, ROCm, GPU programming, HPC, large compute clusters, PyTorch, JAX or Triton would be particularly valuable.

This is an opportunity to own a foundational part of a new AI compute platform, working directly between compiler output, hardware execution and distributed systems.