Staff Software Engineer, ML Infra & Distributed Systems

tubitv; United States; Hybrid; USD 227,000 - 325,000; Full time

Benefits offered by this job

Annual bonus

Long-term incentives

Health, dental and vision coverage

Insurance

Retirement savings

Paid time off

12 weeks paid parental bonding leave

Monthly wellness reimbursement

Job summary

tubiTV is seeking a senior infrastructure leader to design and evolve a distributed ML inference platform enabling low-latency, high-throughput serving for recommendations, search, and content understanding. You will drive architecture with engineering and product teams across the stack.

You will mentor engineers in system design, incident response, and AI tooling, while collaborating on long-term platform direction with partners and stakeholders in a hybrid work setup.

Qualifications

• Experience designing and operating distributed systems in a modern backend language; JVM language familiarity is useful.
• Strong experience with AWS or a comparable cloud platform and large-scale, low-latency online services.
• Familiarity with SQL and NoSQL databases, message brokers, and caching systems.
• Experience with containers, such as Docker or Kubernetes.

Responsibilities

• Design and build scalable distributed services for high-throughput, low-latency ML inference.
• Develop reusable infrastructure for applications including personalization, search, advertising, and exploration.
• Collaborate with ML engineers to identify platform constraints and shape scalable solutions.
• Use operational data to improve latency, cost, and efficiency, and lead refactoring when needed.
• Help define long-term ML infrastructure architecture with product and engineering partners.
• Mentor engineers in system design, incident response, interviewing, and AI coding tools.

Skills

Distributed systems design

AWS

SQL/NoSQL

Docker/Kubernetes

Incident response leadership

Tools

Docker

Kubernetes

Akka

Job description

Role overview

Build and evolve the distributed infrastructure that lets machine-learning teams serve models with low latency and high throughput. The platform supports applications such as recommendations, search, and content understanding, and includes reusable serving components, feature and vector stores, and experimentation capabilities. The role combines hands‑on system design with technical leadership across engineering and product teams.

Responsibilities

• Design and build scalable distributed services for high-throughput, low-latency ML inference.
• Develop reusable infrastructure for applications including personalization, search, advertising, and exploration.
• Work with ML engineers to identify platform constraints and shape practical, scalable solutions.
• Use operational data to improve latency, cost, and efficiency, and lead broader refactoring when needed.
• Help define the long-term architecture for ML infrastructure with product and engineering partners.
• Mentor engineers in system design, incident response, interviewing, and effective use of AI coding tools.

Requirements

• Experience designing and operating distributed systems in a modern backend language; Scala or another JVM language is useful.
• Strong experience with AWS or a comparable cloud platform and large-scale, low-latency online services.
• Familiarity with SQL and NoSQL databases, message brokers, and caching systems.
• Experience with containers, such as Docker or Kubernetes.
• Experience leading the response and resolution of multiple major production incidents.

Nice to have

• Experience with inference engines, vector or feature stores, model-training orchestration, or ML serving systems is useful.
• Background in recommender systems, search, autocomplete, advertising ML, or analysis of A/B test results is also relevant.
• Experience with Akka, Erlang, Elixir, or Go is a plus.

Benefits and work setup

The position is described as hybrid.

• Annual discretionary bonus
• Long‑term incentives
• Health, dental and vision coverage
• Insurance
• Retirement savings
• Paid time off
• 12 weeks of paid parental bonding leave
• Monthly wellness reimbursement

The listed annual pay range for high-cost U.S. labor markets is $227,200–$324,500; final compensation depends on location, education, skills, and experience.