Senior ML Infra Engineer for Scalable Conversational AI

Whereby; Cambridge; Hybrid; GBP 90,000 - 120,000; Full time

Benefits offered by this job

Mental health support

Financial wellness resources

Healthcare (medical, dental, vision)

Job summary

Roku is seeking a Senior Software Engineer to strengthen the Entertainment Assistant team. You will own production ML and infrastructure across ranking, model delivery, evaluation and LLM behavior, collaborating with ML, product, data and platform partners.

This hands-on role spans software architecture, ML lifecycle decisions and production operations to improve training, evaluation, deployment, and observability of models at scale.

Qualifications

• Strong production software engineering experience with distributed services.
• Meaningful ownership of ML infrastructure or production model delivery.
• Practical understanding of ML lifecycle: training, evaluation, versioning, deployment, online validation, monitoring.

Responsibilities

• Own fulfilment-ranking pipelines from training to online validation.
• Build/offline evaluation platform with LLM-driven checks.
• Develop LLM agent with tool routing and caching.
• Design caching to reduce latency and cost.

Skills

ML infrastructure

Distributed systems

Production software engineering

Latency optimization

Education

CS or related degree

Tools

Go

Python

Kubernetes

AWS

GCP

Spark

Kafka

Job description

Roku is seeking a Senior Software Engineer to strengthen the Entertainment Assistant team. You will own production ML and infrastructure across ranking, model delivery, evaluation and LLM behavior, collaborating with ML, product, data and platform partners.

This hands-on role spans software architecture, ML lifecycle decisions and production operations to improve training, evaluation, deployment, and observability of models at scale.