Senior Real-Time ML Infrastructure Engineer

Disney Entertainment & Sports LLC; Glendale (CA); On-site; USD 144,000 - 193,000; Full time

Job summary

ESPN is seeking a senior data engineer to build and run the distributed data and ML infrastructure behind real-time sports personalization and live experiences.

You will design high-throughput batch and streaming pipelines, work with Kafka/Kinesis, Spark/Flink, and cloud-native stacks (AWS, Kubernetes), collaborating with ML and data teams to improve latency, data freshness, and overall system reliability.

Qualifications

• 5+ years of production experience building data-intensive or ML-adjacent systems.
• Experience operating large-scale data or ML systems in production.
• Strong fundamentals in distributed systems and data processing architectures.
• Hands-on with streaming and batch technologies such as Kafka, Kinesis, Spark, Flink.
• Proficiency in Python and working knowledge of Java, Scala, Go, or C++.
• Experience operating in cloud-native environments including AWS, containers, Kubernetes, and IaC tools.
• Familiarity with observability and best practices for production systems.
• Strong collaboration skills across engineering and data teams.
• Experience supporting real-time personalization, recommendation, or analytics systems.
• Familiarity with feature stores, event-driven architectures, and real-time ML pipelines.
• Exposure to ML infrastructure concepts including inference pipelines, data validation, and model lifecycle tooling.
• Experience optimizing data systems for latency, throughput, and cost efficiency.
• Understanding of experimentation platforms and data instrumentation for online systems.

Responsibilities

• Build and maintain high-throughput batch and streaming data pipelines for ML, analytics, and real-time decisioning.
• Implement data ingestion, enrichment, aggregation, and transformation using modern distributed data frameworks.
• Deliver pipelines that meet latency, reliability, and data quality needs for downstream ML and product teams.
• Develop and operate systems for real-time feature computation and delivery used by online ML services.
• Partner with applied ML and data science teams to enable production ML workflows through reliable data systems.
• Work with feature stores and event-driven architectures to keep offline and online data consistent.
• Improve data freshness, schema evolution, and backward compatibility for streaming environments.
• Build and operate ML-adjacent services such as inference inputs, feature APIs, and data access layers.
• Contribute to scalable service patterns including autoscaling, rollout strategies, and resiliency mechanisms.
• Collaborate with platform and SRE teams to improve availability, performance, and cost efficiency.
• Instrument data and ML infrastructure with metrics, logging, and alerting for production operations.
• Participate in on-call rotations and incident response for data and ML platforms.
• Diagnose and remediate data pipeline failures, performance regressions, and operational risks.
• Participate in design reviews, code reviews, and technical discussions.
• Follow established platform standards and contribute incremental improvements over time.

Skills

Distributed systems

Data processing

Kafka/Kinesis

Spark/Flink

Python

Java/Scala/Go/C++

AWS

Kubernetes

IaC tools

Observability

Team collaboration

Real-time analytics

Education

Bachelor’s or Master’s degree in Computer Science, Data Engineering, ML, or related field

Tools

Kafka

Kinesis

Spark

Flink

Java

Scala

Go

C++

AWS

Kubernetes

IaC tools

Feature stores

Job description

ESPN is seeking a senior data engineer to build and run the distributed data and ML infrastructure behind real-time sports personalization and live experiences.

You will design high-throughput batch and streaming pipelines, work with Kafka/Kinesis, Spark/Flink, and cloud-native stacks (AWS, Kubernetes), collaborating with ML and data teams to improve latency, data freshness, and overall system reliability.