Software Engineer II - AI Performance

The Walt Disney Company; New York (NY); On-site; USD 125,000 - 167,000; Full time

Job summary

The Walt Disney Company in New York, NY is seeking a Software Engineer II on the Reliability Insights team to design and build data pipelines and dashboards that measure AI reliability and incident response.

You will bridge backend engineering with data science, partner with SRE and product teams, and help reduce alert fatigue while delivering clear metrics and actionable insights to leadership.

Qualifications

• 3+ years of backend and data pipeline engineering experience.
• Experience with data platforms and distributed processing tools to analyze telemetry data.
• Understanding of model evaluation concepts, such as precision, recall, drift, and ground-truth labeling systems.

Responsibilities

• Design, build, and maintain automated feedback pipelines that tie AI model predictions to incident outcomes.
• Develop evaluation frameworks to track precision, recall, and drift over time with automated alerts.
• Build end-to-end data pipelines linking AI outputs to incident lifecycles and business metrics.
• Design and deploy scalable APIs, metrics layers, and dashboards for leadership and engineers.
• Capture structured user feedback to continuously feed real-world usage signals into models.

Skills

Backend API development

Data pipelines

Distributed processing

Model evaluation concepts

GitHub & Docker

AWS / EKS

AI-assisted development tools

Analytical thinking

Cross-functional collaboration

Education

Bachelor’s degree in Computer Science, Engineering, or equivalent experience

Tools

FastAPI / Flask

PySpark

SQL

Pandas

Databricks

Snowflake

Job description

Job ID 10148555 Location New York, New York, United States Business Disney Entertainment and ESPN Product & Technology Date posted Oct. 20, 2026

Disney Entertainment and ESPN Product & Technology

Technology is at the heart of Disney’s past, present, and future. Disney Entertainment and ESPN Product & Technology is a global organization of engineers, product developers, designers, technologists, data scientists, and more – all working to build and advance the technological backbone for Disney’s media business globally.

The team marries technology with creativity to build world-class products, enhance storytelling, and drive velocity, innovation, and scalability for our businesses.We are Storytellers and Innovators. Creators and Builders. Entertainers and Engineers. We work with every part of The Walt Disney Company’s media portfolio to advance the technological foundation and consumer media touch points serving millions of people around the world.

Here are a few reasons why we think you’d love working here:

• Building the future of Disney’s media: Our Technologists are designing and building the products and platforms that will power our media, advertising, and distribution businesses for years to come.
• Reach, Scale & Impact: More than ever, Disney’s technology and products serve as a signature doorway for fans' connections with the company’s brands and stories. Disney+. Hulu. ESPN. ABC. ABC News…and many more. These products and brands – and the unmatched stories, storytellers, and events they carry – matter to millions of people globally.
• Innovation: We develop and implement groundbreaking products and techniques that shape industry norms and solve complex and distinctive technical problems.

Product Engineering is a unified team responsible for the engineering of Disney Entertainment & ESPN digital and streaming products and platforms. This includes product engineering, media engineering, quality assurance, engineering behind personalization, commerce, lifecycle, and identity.

The Observability & Insights group ensures that Disney Streaming’s distributed systems are reliable, performant, and transparent. We build the telemetry, dashboards, alerting, insights pipelines, and developer experience tooling that enable engineers across the organization to understand system health and take action quickly.

Job Summary:

As a Software Engineer II on the Reliability Insights team, you will design and build data and reporting pipelines that evaluate and prove the real-world performance, precision, and business impact of Disney’s operational AI and observability tools. You will bridge backend engineering and data-driven insights, transforming raw telemetry, AI agent interactions, and operational logs into clear metrics around AI reliability, model drift, and incident mitigation.

In this role, you will build automated ground-truth pipelines and model evaluation frameworks that quantify how effectively AI agents and anomaly models perform in production. You will capture end user feedback, correlate AI signals with incident resolution timelines (e.g., Mean Time to Mitigation), and build single pane dashboards for engineering and leadership. Partnering closely with SRE, product, and platform engineering teams, you will eliminate visibility gaps, reduce alert noise, and establish clear standards for AI model performance across Disney+, Hulu, and ESPN.

Responsibilities:

• Design, build, and maintain automated feedback pipelines that capture operational outcomes (confirmed incidents, false positives, dismissed alerts) and correlate them directly back to AI model predictions and agent actions.
• Develop programmatic evaluation frameworks to track model precision, recall, and drift over time, establishing automated threshold alerts to proactively trigger model tuning and prevent alert fatigue.
• Build end-to-end data pipelines linking AI outputs to incident lifecycles, quantifying business impact through metrics such as Service to Incident correlation, notification success rates, and Mean Time to Mitigation (MTTM).
• Design and deploy scalable APIs, metrics layers, and single-pane model health dashboards that present clear AI ROI, operational health, and performance narratives to leadership and technical stakeholders.
• Build mechanisms to capture structured feedback from end users to continuously feed real-world usage signals back into model improvement cycles.

Basic Qualifications:

• 3+ years of applicable experience in backend and data pipeline engineering, including building scalable APIs (e.g., FastAPI, Flask) and metrics/insight generation platforms.
• Strong experience with data platform engineering and distributed processing tools (e.g., PySpark, SQL, Pandas, Databricks, Snowflake) to process and analyze telemetry data.
• Practical understanding of model evaluation concepts, such as precision, recall, model drift, and ground-truth labeling systems.
• Hands‑on experience with modern development practices, including version control (GitHub), containerization (Docker), and cloud-native deployments (AWS/EKS).
• Proficiency with AI‑assisted development tools (e.g., Cursor, Claude Code) to accelerate engineering velocity.
• Strong analytical skills to translate complex system telemetry and operational logs into meaningful reliability metrics and business insights.
• Excellent cross‑functional collaboration and communication skills, with a track record of partnering with operational teams (SRE, DevOps) to turn raw data into actionable insights

Preferred Qualifications:

• Experience with observability, telemetry, and monitoring platforms (e.g., Datadog, Grafana, Conviva).
• Familiarity with agentic workflows and foundation models (e.g., GPT‑4, Claude) and how to evaluate their output accuracy.
• Knowledge of incident management workflows and SRE metrics (e.g., MTTR, MTTD, alert suppression).
• Prior experience working on data attribution, metrics aggregation, or financial/performance visibility tooling in high scale environments.

Required Education

• Bachelor’s degree in Computer Science, Engineering, or equivalent experience

#TECHATDISNEY

The hiring range for this position in New York, NY is $124,900.00 to $167,400.00 per year. The base pay actually offered will take into account internal equity and also may vary depending on the candidate’s geographic region, job‑related knowledge, skills, and experience among other factors. A bonus and/or long‑term incentive units may be provided as part of the compensation package, in addition to the full range of medical, financial, and/or other benefits, dependent on the level and position offered.

About Disney Entertainment and ESPN Product & Technology:

At Disney Entertainment and ESPN Product & Technology, we’re blending imagination and innovation to reimagine the ways people experience and engage with the world’s most beloved stories and products. Our work is wide-ranging and deeply sophisticated. We create amazing experiences, transform the future of media, and build products and platforms that enable the connection between people everywhere and the stories and sports they love.

Disney’s ability to marry world‑class technology with one‑of‑a‑kind creativity makes us unique. It is at the heart of our past, present, and future. We are Storytellers and Innovators. Creators and Builders. Entertainers and Engineers.

About The Walt Disney Company:

The Walt Disney Company, together with its subsidiaries and affiliates, is a leading diversified international family entertainment and media enterprise that includes three core business segments: Disney Entertainment, ESPN, and Disney Experiences. From humble beginnings as a cartoon studio in the 1920s to its preeminent name in the entertainment industry today, Disney proudly continues its legacy of creating world‑class stories and experiences for every member of the family. Disney’s stories, characters and experiences reach consumers and guests from every corner of the globe. With operations in more than 40 countries, our employees and cast members work together to create entertainment experiences that are both universally and locally cherished.

This position is with Disney Entertainment & Sports LLC, which is part of a business we call Disney Entertainment and ESPN Product & Technology.

Disney Entertainment & Sports LLC is an equal opportunity employer. Applicants will receive consideration for employment without regard to race, religion, color, sex, sexual orientation, gender, gender identity, gender expression, national origin, ancestry, age, marital status, military or veteran status, medical condition, genetic information or disability, or any other basis prohibited by federal, state or local law. Disney champions a business environment where ideas and decisions from all people help us grow, innovate, create the best stories and be relevant in a constantly evolving world.

DISABILITY ACCOMMODATION FOR EMPLOYMENT APPLICATIONS

The Walt Disney Company and its Affiliated Companies are Equal Employment Opportunity employers and welcome all job seekers including individuals with disabilities and veterans with disabilities. If you have a disability and believe you need a reasonable accommodation in order to search for a job opening or apply for a position, visit the Disney candidate disability accommodations FAQs. We will only respond to those requests that are related to the accessibility of the online application system due to a disability.

Explore Disney Careers and the Life at Disney blog to learn about all the amazing opportunities waiting to be discovered at The Walt Disney Company.

Be Part of the Story

There are many different brands and businesses to explore. Once you've found the opportunity that is right for you, take the next step by completing your application.

The Next Chapter

Once you've applied, you will receive an email allowing you to access your candidate dashboard. Create your login and make sure to check your dashboard often to see your application progress.

With a globally recognizable skyline and 8.4 million people who call it home, New York City is bursting with diversity and excitement – from iconic attractions to hidden gems waiting to be uncovered.

Our senior executives bring tremendous experience, visionary thinking and a shared commitment to excellence, creativity and innovation to the day to day operation of the company.

Explore our commitments and our work to create a better world through our stories, experiences, operations, and philanthropy.

At Disney, we want everyone to belong and thrive. Creating a welcoming and respectful environment for our employees and guests is core to our company culture and our business. We strive to build supportive work environments that drive innovation and reinforce a culture in which all employees feel welcome, respected, and valued.

Heroes Work Here reflects the long history of respect and appreciation Disney has for the U.S. Armed Services. We recognize the commitment and dedication it takes to serve your country, both as military personnel and military spouses, and value the leadership skills and sense of purpose it has instilled in you.

Heroes Work Here

Heroes Work Here reflects the long history of respect and appreciation Disney has for the U.S. Armed Services. We recognize the commitment and dedication it takes to serve your country, both as military personnel and military spouses, and value the leadership skills and sense of purpose it has instilled in you.