ML Runtime Engineer
Location: Silicon Valley, California, USA
About the company
A venture backed AI compute start up founded by a team with a track record of shipping high performance silicon. They design silicon, systems and software as one product, combining general purpose processing, AI acceleration and scalable memory in a single architecture that runs from edge devices to data centers.
The role
You will develop and optimize the runtime that executes ML models on the company’s own accelerator hardware. That means owning how memory is allocated, how data moves between host and device, and how workloads are scheduled and run efficiently, working with open standard APIs where they apply.
What you’ll do
- Design and build core runtime components in modern C and C++ for ML and compute workloads on hardware accelerators
- Build the mechanisms for device and context initialization, resource and memory allocation, buffer lifetime management, command submission, synchronization and asynchronous execution
- Optimize data movement between host and device, memory utilization, execution scheduling, and end to end latency and throughput
- Profile, diagnose and resolve bottlenecks across the runtime, driver, operating system and accelerator execution path
- Integrate runtime services with compiler outputs, device drivers, kernel interfaces and the hardware architecture
- Analyze representative ML workloads and benchmarks, and use measurements to guide improvements and regression testing
- Design robust multithreaded systems with sound synchronization, concurrency control, inter process communication and error handling
- Support portability and interoperability across hardware platforms and open standard runtime APIs
- Work with hardware, compiler, kernel, driver, architecture and ML software teams to bring up, validate and optimize new accelerator features
- Improve runtime observability, diagnostics, test coverage, reliability and maintainability
What you’ll bring
- 7 to 15 years of relevant software engineering experience, with substantial hands on systems software development
- Strong C and C++ skills, including debugging complex codebases, memory management, concurrency and performance sensitive design Page 9 of 14 Tsavorite Client Profile
- Proven experience developing or maintaining a runtime, execution engine, device runtime, accelerator software stack or a comparable low level systems component
- Strong Linux systems programming: processes and threads, virtual memory, system calls, IPC, synchronization primitives and Linux debugging
- Practical grasp of multithreaded and asynchronous execution, including race conditions, deadlocks and contention
- Experience with performance profiling and debugging tools, and the ability to isolate bottlenecks and validate improvements with measurements
- Experience with hardware accelerators or heterogeneous compute such as GPUs, AI and ML accelerators, DSPs or other specialized processors
- Understanding of memory hierarchy and data movement, including host and device memory, buffers, allocation strategies and resource lifetime
- BS or MS in Computer Engineering, Electrical Engineering, Computer Science or a related field
Nice to have
- MS degree
- Hands on experience with open standard compute or runtime APIs
Why join
You would take real ownership of a core layer of a new AI platform, working alongside hardware, compiler, kernel and driver engineers at an early stage company where individual engineers have a lot of influence. The package includes competitive compensation, health insurance for you and eligible dependents, and stock options.