Lead Machine Learning Inference Engineer, Advertising

Greenlever; Austin (TX); Hybrid; USD 246,500 - 486,100; Full time

Benefits offered by this job

Health insurance

Equity awards

Life insurance

Disability benefits

Parental leave

Wellness benefits

Paid time off

Global mental health resources

401(k)/pension

Job summary

Roku is seeking a Lead Machine Learning Inference Engineer, Advertising, to architect and lead a state-of-the-art inference platform with low latency and high throughput for advertising workloads. You will mentor engineers, drive cross-functional initiatives, and push innovations in ML serving and HPC techniques.

The role emphasizes leadership, scalable systems, and collaboration across global teams, with a hybrid work model and strong compensation including health and equity benefits.

Qualifications

• MS or above in CS, ECE, or a related field.
• 10+ years of experience in developing and deploying large-scale, distributed systems, with at least 5 years in a leadership or technical lead role.
• Strong programming skills in high-performance languages.
• Deep understanding of inference frameworks and ML system deployment.
• Contributions to open-source ML or systems projects.

Responsibilities

• Lead the design and development of a SOTA Inference platform
• Oversee the development of monitoring, observability, and other tooling to ensure system and model performance, reliability, and scalability of online inference services
• Identify and resolve system inefficiencies, performance bottlenecks, and reliability issues, ensuring optimized end-to-end performance
• Stay at the forefront of advancements in inference frameworks, ML hardware acceleration, and distributed systems, and incorporate innovations where and when they are impactful

Skills

ML serving

Distributed systems

GPU acceleration

High-performance languages

Leadership

Education

MS in CS or ECE

Tools

ML frameworks

HPC tools

Job description

Lead Machine Learning Inference Engineer, Advertising

Software Engineering | San Jose, United States | ID: 10787

Roku Is Changing How The World Watches TV

Roku is the #1 TV streaming platform in the U.S., Canada, and Mexico, and we've set our sights on powering every television in the world. Roku pioneered streaming to the TV. Our mission is to be the TV streaming platform that connects the entire TV ecosystem. We connect consumers to the content they love, enable content publishers to build and monetize large audiences, and provide advertisers unique capabilities to engage consumers.

About The Team

The Advertising Performance group focuses on performance for all participants in the Advertising ecosystem – Advertisers, Publishers, and Roku. The systems and solutions span multiple disciplines and technologies to perform real-time multi-objective optimization across large-scale distributed systems with low latency. We use Machine Learning, Reinforcement Learning, AI, Control and Optimization Systems, and Auction Dynamics to solve a large set of complex problems. At the core of this is our Machine Learning and Inference Platform that powers the entire landscape.

About The Role

In this role, you will architect, design, and lead the development of a SOTA Inference platform that can handle Advertising-level low latencies, scale, throughput, and availability with optimizations that span across hardware, software, and models. We’re looking for a strong technical leader with deep experience in ML serving, high-performance computing, and industry standard frameworks – someone excited to mentor engineers, innovate at scale, and shape the future of machine learning at Roku.

For California Only - The estimated annual salary for this position is between $246,500 - $486,100 annually. Compensation packages are based on factors unique to each candidate, including but not limited to skill set, certifications, and specific geographical location. This role is eligible for health insurance, equity awards, life insurance, disability benefits, parental leave, wellness benefits, and paid time off.

What you’ll be doing

• Lead the design and development of a SOTA Inference platform
• Oversee the development of monitoring, observability, and other tooling to ensure system and model performance, reliability, and scalability of online inference services
• Identify and resolve system inefficiencies, performance bottlenecks, and reliability issues, ensuring optimized end-to-end performance
• Stay at the forefront of advancements in inference frameworks, ML hardware acceleration, and distributed systems, and incorporate innovations where and when they are impactful

We’re excited if you have

• M.S. or above in CS, ECE, or a related field
• 10+ years of experience in developing and deploying large-scale, distributed systems, with at least 5 years in a leadership or technical lead role
• Strong programming skills in high-performance languages
• Deep understanding of inference frameworks and ML system deployment
• Proven experience optimizing performance for large-scale machine learning systems, including a deep knowledge of SOTA model optimizations, hardware-software co-design, GPU acceleration, and HPC techniques
• Excellent communication and collaboration skills
• Experience leading teams working on high-throughput, low-latency ML serving systems
• Experience collaborating with and leading global, cross-functional teams
• Contributions to open-source ML or systems projects

What's Roku's approach to hybrid working?

Roku fosters an inclusive and collaborative environment where teams generally work in the office Monday through Thursday. Fridays are generally flexible for remote work, except for employees whose specific roles or assigned office location require five days' a week attendance.

What are some of the benefits?

• Health insurance
• Equity awards
• Life insurance
• Disability benefits
• Parental leave
• Wellness benefits
• Paid time off
• Global access to mental health and financial wellness support and resources
• Statutory and voluntary benefits which may include healthcare (medical, dental, and vision)
• Life
• Accident
• Disability
• Commuter
• Retirement options (401(k)/pension)

Accommodations

Roku welcomes applicants of all backgrounds and provides reasonable accommodations and adjustments in accordance with applicable law. If you require reasonable accommodation at any point in the hiring process, please direct your inquiries to [e-mail].

What should I know about Roku's culture?

Roku is a great place for people who want to work in a fast-paced environment where everyone is focused on the company's success rather than their own. We try to surround ourselves with people who are great at their jobs, who are easy to work with, and who keep their egos in check. We appreciate a sense of humor. We believe a fewer number of very talented folks can do more for less cost than a larger number of less talented teams. We're independent thinkers with big ideas who act boldly, move fast and accomplish extraordinary things through collaboration and trust. In short, at Roku you'll be part of a company that's changing how the world watches TV.

We have a unique culture that we are proud of. We think of ourselves primarily as problem-solvers, which itself is a two-part idea. We come up with the solution, but the solution isn't real until it is built and delivered to the customer. That penchant for action gives us a pragmatic approach to innovation, one that has served us well since 2002.

By providing your information, you acknowledge that you want Roku to contact you about job roles, that you have read Roku's Applicant Privacy Notice, and understand that Roku will use your information as described in that notice. If you do not wish to receive any communications from Roku regarding this role or similar roles in the future, you may unsubscribe at any time by emailing [e-mail].