Lead AI-Driven Streaming Observability Engineer

Disney Cruise Line - The Walt Disney Company; New York (NY); On-site; Confidential; Full time

Job summary

Disney Entertainment and ESPN Product & Technology in New York is seeking a Lead Software Engineer to design and build AI-driven systems for a large-scale streaming ecosystem.

You will partner with engineering, product, and platform teams to embed intelligence into operational workflows, improve resilience, and enhance subscriber experience at scale.

The role emphasizes real-time telemetry, automated detection, and responsible deployment in a fast-paced, AI-native environment.

Qualifications

• 7+ years in backend development, including AI-powered apps and scalable APIs.
• Proficiency in Spark, PySpark, Python or similar.
• Experience building production-scale systems in distributed environments.
• Strong AI/ML engineering experience with foundation models via LangChain or LangGraph.
• Familiarity with PyTorch or TensorFlow for model training.
• Proficiency with GitHub, Docker, and AWS/EKS.
• Strong API design and microservices knowledge.
• Excellent collaboration and communication across teams.

Responsibilities

• Architect, design, and implement distributed systems leveraging real-time telemetry and AI-driven detection to improve health and reliability.
• Define architectural patterns and scaling strategies for agentic AI systems with foundation models.
• Develop end-to-end data and decisioning pipelines transforming telemetry into insights.
• Create and deploy scalable APIs and services delivering predictive signals to teams and stakeholders.
• Partner cross-functionally to embed intelligence into incident response and subscriber experience workflows.
• Drive observability, reliability, and developer productivity through applied AI.
• Define and drive engineering standards, code quality, and CI/CD, mentoring engineers.

Skills

Backend development

Spark / PySpark

Python

AI/ML engineering

Foundation models

LangChain / LangGraph

PyTorch / TensorFlow

AI development tools

Git / GitHub

Docker

AWS / EKS

API design

Microservices

SDLC practices

Analytical skills

Collaboration/communication

Education

Bachelor's degree in Computer Science, Engineering, or equivalent experience

Tools

Datadog

Grafana

Conviva

PySpark

Pandas

Databricks

Snowflake

Job description

Disney Entertainment and ESPN Product & Technology in New York is seeking a Lead Software Engineer to design and build AI-driven systems for a large-scale streaming ecosystem.

You will partner with engineering, product, and platform teams to embed intelligence into operational workflows, improve resilience, and enhance subscriber experience at scale.

The role emphasizes real-time telemetry, automated detection, and responsible deployment in a fast-paced, AI-native environment.