Runtime Engineer

Oho Group; San Francisco (CA); On-site; USD 180,000 - 240,000; Full time

Job summary

Oho Group is seeking an AI Accelerator Software Engineer — Runtime & Execution to help build the software stack for a new accelerator architecture. The role focuses on turning compiled programs into efficient execution on hardware, interfacing with compiler, device-level software, and workload execution to address synchronization, memory, scheduling, and performance challenges.

You will work on core components of the runtime, manage program dispatch and asynchronous execution, and collaborate

Qualifications

• Experience with runtimes, drivers, compilers or performance-sensitive infrastructure.
• Understanding of concurrency, synchronization and memory management.
• Strong debugging skills across software and hardware boundaries.

Responsibilities

• Build core components of an accelerator runtime and execution engine.
• Manage program dispatch, synchronization and asynchronous execution.
• Design memory allocation, movement and lifetime-management mechanisms.
• Implement interfaces between compiled programs and underlying devices.
• Support multi-device execution and communication.
• Profile runtime behavior and remove latency, memory and concurrency bottlenecks.
• Develop debugging, tracing and performance-analysis capabilities.
• Collaborate closely with compiler, architecture and framework engineers.

Skills

Runtimes

Concurrency

Memory management

Debugging

Performance infrastructure

CUDA

ROCm

OpenCL

SYCL

LLVM

Tools

CUDA

ROCm

OpenCL

SYCL

MLIR

LLVM

Job description

AI Accelerator Software Engineer — Runtime & Execution

We’re working with an AI hardware company building the software stack for a new accelerator architecture.

This role owns the layer that turns compiled programs into efficient execution on the hardware.

You’ll work between the compiler, device-level software and workload execution, solving synchronization, memory, scheduling and performance problems that directly affect real applications.

What you’ll work on

• Build core components of an accelerator runtime and execution engine
• Manage program dispatch, synchronization and asynchronous execution
• Design memory allocation, movement and lifetime-management mechanisms
• Implement interfaces between compiled programs and underlying devices
• Support multi-device execution and communication
• Profile runtime behavior and remove latency, memory and concurrency bottlenecks
• Develop debugging, tracing and performance-analysis capabilities
• Collaborate closely with compiler, architecture and framework engineers

What we’re looking for

• Experience with runtimes, drivers, compilers or performance-sensitive infrastructure
• Understanding of concurrency, synchronization and memory management
• Strong debugging skills across software and hardware boundaries
• Ability to design foundational systems while interfaces are still evolving

Experience with CUDA, ROCm, OpenCL, SYCL, MLIR, LLVM, GPUs, AI accelerators or heterogeneous computing would be particularly relevant.

This is a strong opportunity for someone who wants to define how software actually executes on new hardware.