Portable AI Runtime & Scheduling Engineer

Oho Group; San Francisco (CA); On-site; USD 150,000 - 210,000; Full time

Job summary

Oho Group is seeking a Runtime Engineer to turn optimized compiler output into reliable, high-performance execution for AI workloads. You will design a portable runtime, enabling parallel execution, scheduling, and profiling across diverse hardware.

You will collaborate with compiler and hardware engineers on runtime architecture, profiling, and performance analysis to push the platform forward for real ML workloads and deployment needs.

Qualifications

• 4+ years in compiler, runtime or low-level systems engineering.
• Strong modern C++ proficiency.
• Deep knowledge of concurrency and asynchronous execution.
• Understanding of memory hierarchies and vector/scalar compute.
• Experience with OS kernels, hypervisors or similarly low-level systems.

Responsibilities

• Design a portable runtime for AI workloads.
• Build workload partitioning, parallel execution and kernel scheduling capabilities.
• Prototype new runtime techniques and evaluate them against real hardware.
• Benchmark compiled workloads and diagnose performance bottlenecks.
• Develop profiling tools that inform compiler and runtime improvements.
• Help evolve the platform around real ML workloads and deployment requirements.

Skills

Modern C++

Concurrency

Memory hierarchy

Low-level systems

Tools

CUDA

ROCm

PyTorch

JAX

Triton

GPU programming

Job description

Oho Group is seeking a Runtime Engineer to turn optimized compiler output into reliable, high-performance execution for AI workloads. You will design a portable runtime, enabling parallel execution, scheduling, and profiling across diverse hardware.

You will collaborate with compiler and hardware engineers on runtime architecture, profiling, and performance analysis to push the platform forward for real ML workloads and deployment needs.