Software Engineer, ML Infra & Distributed Systems (Staff & Principal)

Tubi; Toronto; On-site; CAD 164,600 - 235,100; Full time

Benefits offered by this job

Annual discretionary bonus

Flexible Time Off Policy

Parental Leave Program

Monthly wellness reimbursement

Job summary

A leading streaming service is seeking a Staff Software Engineer for its ML Infrastructure team in Toronto. The role focuses on designing low-latency distributed systems and optimizing machine learning application performance. The ideal candidate has over 8 years in distributed systems using Scala or equivalent, strong cloud experience, and a knack for building scalable solutions. This position offers a competitive salary between $164,600 and $235,100 CAD, plus various benefits including a discretionary bonus and wellness reimbursements.

Qualifications

• 8+ years of experience designing and building scalable, distributed systems in a modern backend language.
• Strong experience with AWS or an equivalent cloud platform.
• Experience with online microservices at scale with low‑latency serving.

Responsibilities

• Design and build scalable, high‑throughput, and low‑latency distributed systems using Scala.
• Build reusable components and services that serve various ML applications.
• Mentor other engineers on the team on system design and effective incident management.

Skills

Scala

AWS

SQL

NoSQL

Docker

Kubernetes

Job description

About The Role

As a Staff Software Engineer on the ML Infrastructure team, you will collaborate closely with the Machine Learning and Product teams to build world‑class machine learning inference platforms. These platforms power essential services like personalized recommendations, search, and content understanding across Tubi.

A core responsibility of this team is developing and maintaining low‑latency ML model serving systems that support Deep Learning, LLM, and Search models. This involves building self‑service infrastructure and critical components such as the inference engine, feature store, vector store, and experimentation engine.

You will improve the way we deploy and operate our services and even contribute to open‑source projects. This role grants the architectural freedom to explore new frameworks, lead critical cross‑functional projects, and transform the capabilities of our ML and Product teams.

Responsibilities

• Design and build scalable, high‑throughput, and low‑latency distributed systems using Scala
• Build reusable components and services that serve various ML applications like Personalization, Search, Ads, and Exploration
• Partner closely with ML engineers to understand their challenges and limitations and develop scalable solutions to address them. Proactively recommend solutions to keep our ML Inference stack state of the art
• Take a data‑driven approach to identifying & optimizing latency, cost, and efficiency of our infra. Lead large scale cross‑functional refactorings if necessary
• Mentor other engineers on the team on system design, effective incident management, interviewing, leveraging LLMs for work, etc.
• Collaborate with ML, Product, and cross‑functional engineering teams to define the long‑term vision and architecture for ML Infrastructure at Tubi

Your Background

• 8+ years of experience designing and building scalable, distributed systems in any modern backend language (e.g., Scala, Java, Python, Go, C++); experience with Scala or JVM‑based language is a plus
• Strong experience with AWS or an equivalent cloud platform
• Experience building online microservices at scale with low‑latency serving
• Experience with both SQL (e.g. Postgres) and NoSQL databases (e.g. Cassandra), message brokers (e.g. Kafka), and caches (e.g. Redis)
• Experience with containerization technologies, such as Docker or Kubernetes
• Led the response and resolution efforts for multiple major, large‑scale incidents

Bonus

• Familiarity with the machine learning infrastructure like inference engines (e.g. torschserve, triton, vLLM), vector stores (e.g. LanceDB, FAISS), feature stores (e.g. Feast), ElastiCache, model training orchestration, etc.
• Understanding of ML model training pipelines and model internals. Experience with Recommender Systems, Search, Autocomplete and Ads ML is a plus
• Previous experience with Akka, Erlang, Elixir or Go
• Proficient in data‑driven analysis of complex A/B testing results

About Tubi

Tubi is a free streaming service that entertains over 100 million monthly active users. Tubi offers the world’s largest collection of Hollywood movies and TV shows, thousands of creator‑led stories and hundreds of Tubi Originals made for the most passionate fans. Headquartered in San Francisco and founded in 2014, Tubi is part of Tubi Media Group, a division of Fox Corporation.

Pursuant to local pay disclosure requirements, the pay range for this role, with final offer amount dependent on education, skills, experience, and location is as listed annually below.

Toronto, Canada

$164,600 - $235,100 CAD

This role is also eligible for an annual discretionary bonus, long‑term incentive plan, and various benefits including medical/dental/vision, insurance, vacation/paid time off and other benefits in accordance with applicable plan documents.

For all salaried employees, in lieu of the FOX Vacation policy, Tubi offers a Flexible Time Off Policy to manage all personal matters.

For all full‑time, regular employees, in lieu of FOX Paid Parental Leave, Tubi offers a generous Parental Leave Program, which allows parents twelve (12) weeks of paid bonding leave (top up in Canada) within the first year of birth, adoption, surrogacy, or foster placement of a child in addition to applicable government leave program(s) and FOX’s short‑term disability policy (if applicable). This time is 100% paid through a combination of any applicable government leaves and wage‑replacement programs in addition to contributions made by Tubi.

For all full‑time, regular employees, Tubi offers a monthly wellness reimbursement.

We are an equal opportunity employer and all qualified applicants will receive consideration for employment without regard to race, color, religion, sex, national origin, gender identity, disability, protected veteran status, or any other characteristic protected by law. We will consider for employment qualified applicants with criminal histories consistent with applicable law.