Lead Big Data Platform Engineer

The Walt Disney Company; San Francisco (CA); On-site; USD 173,000 - 232,000; Full time

Job summary

The Walt Disney Company is seeking a Lead Software Engineer for Disney Entertainment and ESPN Technology in San Francisco. You will help build large-scale data platforms and backend services, enabling real-time and batch processing across Disney's media ecosystem.

Expect to drive architecture decisions, mentor teams, and collaborate with researchers and product managers in an open, innovative environment. This role emphasizes performance, scalability, and cutting-edge technologies.

Qualifications

• Bachelor's degree in Computer Science, Electronics Engineering, or related field.
• At least 7 years of professional programming and design experience in Scala, Java, Python, etc.
• 3+ years of big data development experience with Databricks, Spark, Hive, SingleStore, Airflow, and AWS.
• Strong knowledge of system and application design and architecture.
• Experience with processing large amounts of data at the petabyte level.
• Demonstrated ability with cloud infrastructure technologies, including Terraform, Kubernetes, Spinnaker, etc.
• Excellent communication skills and project leadership experience.

Responsibilities

• Contribute to a team of big data and backend engineers to deliver high value projects.
• Build components of large-scale data platform for batch and real-time processing, and own features of big data applications to fit evolving business needs.
• Build next-gen cloud based big data infrastructure for batch and streaming data applications, and continuously improve performance, scalability and availability.
• Balance architectural and design considerations such as performance, scalability, reusability and flexibility issues.
• Advocate the best engineering practices, including the use of design patterns, CI/CD, code review and automated test.
• Collaborate with researchers to productize cutting edge innovations at scale.
• Collaborate with program managers, product managers, SDET, and researchers in an open and innovative environment.

Skills

Scala/Java/Python

Big data development

Cloud infra

Communication/leadership

Performance/scalability

Education

Bachelor's degree

Tools

Databricks

Spark

Hive

SingleStore

Airflow

AWS

Terraform

Kubernetes

Spinnaker

Job description

The Walt Disney Company is seeking a Lead Software Engineer for Disney Entertainment and ESPN Technology in San Francisco. You will help build large-scale data platforms and backend services, enabling real-time and batch processing across Disney's media ecosystem.

Expect to drive architecture decisions, mentor teams, and collaborate with researchers and product managers in an open, innovative environment. This role emphasizes performance, scalability, and cutting-edge technologies.