Senior / Staff ML Training Optimization Engineer

Waabi; United States; Hybrid; USD 141,000 - 249,000; Full time

Benefits offered by this job

Competitive compensation and equity awards

Health and Wellness benefits (Medical, Dental, Vision)

Unlimited Vacation

Flexible hours and Work from Home support

Daily drinks, snacks and catered meals

Team building activities and social events

World-class facility with gym and collaborative spaces

Job summary

Waabi is looking for a skilled professional to develop distributed training frameworks that enhance the stability and efficiency of our systems. This role requires at least 4 years of relevant experience and proficiency in Python, C++, or Rust. You will work closely with researchers to advance our self-driving technologies and create innovative solutions.

We offer a competitive salary range of $141,000 to $249,000, health benefits, unlimited vacation, and flexible work hours. Join us and be part of a diverse and inclusive team that is redefining the future!

Qualifications

• 4+ years of industry experience in relevant fields.
• Solid coding proficiency in Python, C++, or Rust.
• Experience in deep learning frameworks such as PyTorch or Jax.
• Skilled in profiling CPU and GPU code using tools like PyTorch Profiler.

Responsibilities

• Build standardized distributed training frameworks.
• Profile model runtime and memory to find performance bottlenecks.
• Work with researchers and ML engineers on best practices.

Skills

Python

C++

Rust

Deep learning frameworks

Profiling tools

Education

MS/PhD or Bachelor’s in Computer Science or related

Tools

PyTorch

Jax

NVIDIA Nsight

Job description

Responsibilities

• Build standardized distributed training frameworks for research and production, drive our training towards new levels of stability and efficiency.
• Comprehensively profile model runtime and memory to pinpoint performance bottlenecks.
• Identify and evaluate emerging technologies that can be adopted into Waabi’s training and inference frameworks, such as designing new CUDA kernels, quantization‑aware training and inference, and compilation/deployment techniques.
• Work with researchers and ML engineers on best‑practices for optimal resource usage.
• Create and improve tooling and dashboards to ensure broad adoption of your work.

Qualifications

• MS/PhD or Bachelor’s degree with a minimum of 4years of industry experience in Computer Science, Robotics, and/or similar technical field(s) of study.
• Solid coding proficiency in Python, C++ or Rust.
• Experience in deep learning frameworks such as PyTorch or Jax.
• Skilled in profiling CPU and GPU code using tools such as PyTorch Profiler and NVIDIA Nsight.
• Open‑minded and collaborative team player with willingness to help others.
• Passionate about self‑driving technologies, solving hard problems, and creating innovative solutions.

Bonus / Nice to Have

• Experience in identifying when custom CUDA kernels are needed and implementing them.
• Experience with Bazel in a monorepo environment and integrating third‑party packages into development environments.
• Experience with Kubernetes‑based training platforms.

The U.S. yearly salary range for this role is $141,000 – $249,000, plus equity incentives and an annual performance bonus.

Perks / Benefits

• Competitive compensation and equity awards.
• Health and Wellness benefits encompassing Medical, Dental and Vision coverage.
• Unlimited Vacation.
• Flexible hours and Work from Home support.
• Daily drinks, snacks and catered meals (when in office).
• Regularly scheduled team building activities and social events both on‑site, off‑site and virtually.
• World‑class facility that includes a gym, games room (ping‑pong table, video game consoles, board games, etc.), multiple collaborative working spaces and a gorgeous patio (when in office).
• As we grow, this list continues to evolve!

Waabi is an equal opportunity employer that celebrates diversity and is committed to creating a supportive, inclusive, and accessible environment for all employees. If reasonable accommodation is needed to participate in the job application or interview process please let our recruiting team know.