Lead ML Inference Engineer, Advertising

Roku, Inc.; Austin (TX); Hybrid; USD 150,000 - 200,000; Full time

Benefits offered by this job

Global access to mental health support

401(k)/pension options

Flexible remote work on Fridays

Job summary

Roku, Inc. is seeking a technical leader to architect and design a state-of-the-art inference platform. The role involves leading the development of systems ensuring low latencies and high performance while mentoring engineers.

Candidates should have over 10 years of experience in large-scale systems, strong programming skills, and a deep understanding of ML frameworks. Roku promotes a hybrid work model and offers various employee benefits.

Qualifications

• At least 10 years of experience in developing large-scale distributed systems.
• 5+ years in a leadership or technical lead role.
• Experience with GPU acceleration and HPC techniques.

Responsibilities

• Lead the design and development of an inference platform.
• Ensure system performance, reliability, and scalability of online inference services.
• Identify and resolve performance bottlenecks.

Skills

Experience in developing and deploying large-scale systems

Strong programming skills in high-performance languages

Deep understanding of inference frameworks

Experience optimizing machine learning systems

Excellent communication skills

Experience leading high-throughput ML systems

Collaboration with cross-functional teams

Education

M.S. or above in Computer Science, ECE, or a related field

Job description

About the team

The Advertising Performance group focuses on performance for all participants in the Advertising ecosystem—Advertisers, Publishers, and Roku. The systems and solutions span multiple disciplines and technologies to perform real‑time multi‑objective optimization across distributed systems at large scale and with low latency. We use Machine Learning, Reinforcement Learning, AI, Control and Optimization Systems, and Auction Dynamics to solve a large set of complex problems. At the core of this is our Machine Learning and Inference Platform that powers the entire landscape.

About the role

In this role, you will architect, design, and lead the development of a state‑of‑the‑art inference platform that can handle Advertising‑level low latencies, scale, throughput, and availability with optimizations that span across hardware, software, and models. We’re looking for a strong technical leader with deep experience in ML serving, high‑performance computing, and industry standard frameworks—someone excited to mentor engineers, innovate at scale, and shape the future of machine learning at Roku.

What you’ll be doing

• Lead the design and development of a state‑of‑the‑art inference platform.
• Oversee the development of monitoring, observability, and other tooling to ensure system and model performance, reliability, and scalability of online inference services.
• Identify and resolve system inefficiencies, performance bottlenecks, and reliability issues, ensuring optimized end‑to‑end performance.
• Stay at the forefront of advancements in inference frameworks, ML hardware acceleration, and distributed systems, and incorporate innovations where and when they are impactful.

We’re excited if you have

• M.S. or above in CS, ECE, or a related field.
• 10+ years of experience in developing and deploying large‑scale, distributed systems, with at least 5 years in a leadership or technical lead role.
• Strong programming skills in high‑performance languages.
• Deep understanding of inference frameworks and ML system deployment.
• Proven experience optimizing performance for large‑scale machine learning systems, including a deep knowledge of state‑of‑the‑art model optimizations, hardware‑software co‑design, GPU acceleration, and HPC techniques.
• Excellent communication and collaboration skills.
• Experience leading teams working on high‑throughput, low‑latency ML serving systems.
• Experience collaborating with and leading global, cross‑functional teams.
• Contributions to open‑source ML or systems projects.

Our Hybrid Work Approach

Roku fosters an inclusive and collaborative environment where teams work in the office Monday through Thursday. Fridays are flexible for remote work except for employees whose roles are required to be in the office five days a week or employees who are in offices with a five‑day in‑office policy.

Benefits

Roku is committed to offering a diverse range of benefits as part of our compensation package to support our employees and their families. Our comprehensive benefits include global access to mental health and financial wellness support and resources. Local benefits include statutory and voluntary benefits which may include healthcare (medical, dental, and vision), life, accident, disability, commuter, and retirement options (401(k)/pension). Employees are supported in taking time off, in accordance with local leave policies and other personal needs to support their evolving work and life needs. It’s important to note that not every benefit is available in all locations or for every role. For details specific to your location, please consult your recruiter.

Accommodations

Roku welcomes applicants of all backgrounds and provides reasonable accommodations and adjustments in accordance with applicable law. If you require reasonable accommodation at any point in the hiring process, please direct your inquiries to [e-mail].