Lead ML Inference Platform Architect (Ad Tech)

Greenlever; Austin (TX); Hybrid; USD 246,500 - 486,100; Full time

Benefits offered by this job

Health insurance

Equity awards

Life insurance

Disability benefits

Parental leave

Wellness benefits

Paid time off

Global mental health resources

401(k)/pension

Job summary

Roku is seeking a Lead Machine Learning Inference Engineer, Advertising, to architect and lead a state-of-the-art inference platform with low latency and high throughput for advertising workloads. You will mentor engineers, drive cross-functional initiatives, and push innovations in ML serving and HPC techniques.

The role emphasizes leadership, scalable systems, and collaboration across global teams, with a hybrid work model and strong compensation including health and equity benefits.

Qualifications

• MS or above in CS, ECE, or a related field.
• 10+ years of experience in developing and deploying large-scale, distributed systems, with at least 5 years in a leadership or technical lead role.
• Strong programming skills in high-performance languages.
• Deep understanding of inference frameworks and ML system deployment.
• Contributions to open-source ML or systems projects.

Responsibilities

• Lead the design and development of a SOTA Inference platform
• Oversee the development of monitoring, observability, and other tooling to ensure system and model performance, reliability, and scalability of online inference services
• Identify and resolve system inefficiencies, performance bottlenecks, and reliability issues, ensuring optimized end-to-end performance
• Stay at the forefront of advancements in inference frameworks, ML hardware acceleration, and distributed systems, and incorporate innovations where and when they are impactful

Skills

ML serving

Distributed systems

GPU acceleration

High-performance languages

Leadership

Education

MS in CS or ECE

Tools

ML frameworks

HPC tools

Job description

Roku is seeking a Lead Machine Learning Inference Engineer, Advertising, to architect and lead a state-of-the-art inference platform with low latency and high throughput for advertising workloads. You will mentor engineers, drive cross-functional initiatives, and push innovations in ML serving and HPC techniques.

The role emphasizes leadership, scalable systems, and collaboration across global teams, with a hybrid work model and strong compensation including health and equity benefits.