Senior ML Infrastructure Engineer: AI Pipelines & Scale

Coding Black Females Ltd; Cambridge; Remote; GBP 90,000 - 130,000; Full time

Job summary

Roku is seeking a Senior Software Engineer with strong ML infrastructure experience to own production systems across ranking, model delivery, evaluation and LLM-agent behaviour. You will improve training, evaluation, deployment and observation paths while managing latency, quality, and cost as the product scales.

You will work across ML, product, data and platform teams to turn ambiguous problems into measurable delivery outcomes, building on agent frameworks, routing, and caching to support

Qualifications

• Strong production software-engineering experience
• Ownership of ML infrastructure or production model delivery
• Understanding of ML lifecycle: training, evaluation, deployment, monitoring
• Experience with latency, observability, and cost in production systems
• Collaborative, able to explain trade-offs to partners

Responsibilities

• Own fulfilment-ranking pipelines from training orchestration through deployment and online validation
• Build and operate an offline-evaluation platform with LLm-based checks
• Develop the Roku Entertainment Assistant’s LLM agent including tool routing and caching
• Design caching to reduce latency and cost for high-volume services
• Build observability for ML/LLM systems including latency attribution and cost tracing
• Improve reliability and operability of distributed systems and diagnose cross-service failures
• Drive accountable AI-assisted engineering practices across the team
• Collaborate across ML, product, data and platform to turn ambiguous problems into measurable outcomes

Job description

Roku is seeking a Senior Software Engineer with strong ML infrastructure experience to own production systems across ranking, model delivery, evaluation and LLM-agent behaviour. You will improve training, evaluation, deployment and observation paths while managing latency, quality, and cost as the product scales.

You will work across ML, product, data and platform teams to turn ambiguous problems into measurable delivery outcomes, building on agent frameworks, routing, and caching to support