Sr Data Engineer

ESPN; Santa Monica (CA); On-site; USD 144,000 - 193,000; Full time

Job summary

Disney Entertainment & ESPN Product & Technology is seeking a senior big data engineer to design and maintain Identity Householding Data products and scalable data pipelines. This role focuses on real-time and batch data delivery and requires deep knowledge of distributed systems and cloud data tooling.

You will partner with cross-functional teams to evolve data models, improve performance, and reduce costs while practicing Agile methodologies and on-call support.

Qualifications

• Bachelor's degree in Computer science, Information Systems, Software, Electrical or Electronics Engineering, or comparable field of study, and/or equivalent work experience.
• 5+ years of related big data engineering experience modeling and developing large data pipelines.
• Hands-on experience with distributed systems such as Databricks compute using tools/libraries such as Spark, Presto, Hive to query and process large datasets at the petabyte level.
• Strong Python, Scala/Spark and SQL skills processing big datasets.
• Experience with at least one major MPP or cloud database technology (Snowflake, Redshift, Big Query, Databricks).
• Familiarity with Data Modeling techniques and Data Warehousing standard methodologies and practices.
• Operational experience in multi-system/services production environment as point-of-contact / SME.
• Problem solving skills with strong attention to detail and excellent analytical and communication skills.
• Solid experience with data integration and orchestration toolsets (e.g. Airflow), CI/CD.
• Solid experience with AWS S3.

Responsibilities

• Build and maintain product-driven initiatives to improve and expand Identity and Device data product offerings.
• Enhance/improve the underlying codebase to be extensible, reusable, and maintainable. Increase unit test coverage on the existing codebase.
• Enhance/redesign existing data models with input from Architecture in order to modernize products and/or improve performance and efficiency, and decrease costs.
• Promote and support Agile methodologies such as Scrum, Kanban, and Scrumban by actively participating in regular ceremonies such as stand-up, retrospectives and sprint planning.
• Collaborate with your squad, Product Managers, Designers, QA, Operations, and other stakeholders to understand requirements and articulate technical decisions and outcomes.
• Participate in on-call support
• Participate in code reviews and problem-solving.

Skills

Python

Scala/Spark

SQL

Distributed systems

Education

Bachelor's degree in Computer Science or related field

Tools

Databricks

Spark

Presto

Hive

Airflow

AWS S3

Job description

Technology is at the heart of Disney’s past, present, and future. Disney Entertainment and ESPN Product & Technology is a global organization of engineers, product developers, designers, technologists, data scientists, and more – all working to build and advance the technological backbone for Disney’s media business globally.

The team marries technology with creativity to build world-class products, enhance storytelling, and drive velocity, innovation, and scalability for our businesses.We are Storytellers and Innovators. Creators and Builders. Entertainers and Engineers. We work with every part of The Walt Disney Company’s media portfolio to advance the technological foundation and consumer media touch points serving millions of people around the world.

Here are a few reasons why we think you’d love working here:

Building the future of Disney’s media: Our Technologists are designing and building the products and platforms that will power our media, advertising, and distribution businesses for years to come.

Reach, Scale & Impact: More than ever, Disney’s technology and products serve as a signature doorway for fans' connections with the company’s brands and stories. Disney+. Hulu. ESPN. ABC. ABC News…and many more. These products and brands – and the unmatched stories, storytellers, and events they carry – matter to millions of people globally.

Innovation: We develop and implement groundbreaking products and techniques that shape industry norms, and solve complex and distinctive technical problems.

Ad Platforms is responsible for Disney’s industry-leading ad technology and products – driving advertising performance, innovation, and value in Disney’s sports, news, and entertainment content, across all media platforms.

This role involves building and maintaining our Identity Householding Data products. These products serve as a comprehensive source of truth for enterprise-wide customer identifiers and their groupings. The role requires expert knowledge of building scalable, fault-tolerant data processing pipelines and backend to ensure the reliable delivery of both real-time and batch data.

Responsibilities

• Build and maintain product-driven initiatives to improve and expand Identity and Device data product offerings
• Enhance/improve the underlying codebase to be extensible, reusable, and maintainable. Increase unit test coverage on the existing codebase.
• Enhance/redesign existing data models with input from Architecture in order to modernize products and/or improve performance and efficiency, and decrease costs.
• Promote and support Agile methodologies such as Scrum, Kanban, and Scrumban by actively participating in regular ceremonies such as stand-up, retrospectives and sprint planning.
• Collaborate with your squad, Product Managers, Designers, QA, Operations, and other stakeholders to understand requirements and articulate technical decisions and outcomes.
• Participate in on-call support
• Participate in code reviews and problem-solving.

Basic Qualifications

• Bachelor's degree in Computer science, Information Systems, Software, Electrical or Electronics Engineering, or comparable field of study, and/or equivalent work experience.
• 5+ years of related big data engineering experience modeling and developing large data pipelines
• Hands-on experience with distributed systems such as Databricks compute using tools/libraries such as Spark, Presto, Hive to query and process large datasets at the petabyte level.
• Strong Python, Scala/Spark and SQL skills processing big datasets
• Experience with at least one major MPP or cloud database technology (Snowflake, Redshift, Big Query, Databricks)
• Familiarity with Data Modeling techniques and Data Warehousing standard methodologies and practices
• Operational experience in multi-system/services production environment as point-of-contact / SME
• Problem solving skills with strong attention to detail and excellent analytical and communication skills
• Solid experience with data integration and orchestration toolsets (e.g. Airflow), CI/CD
• Solid experience with AWS S3

Preferred Qualifications

• Demonstrated ability with cloud infrastructure technologies, including Terraform.
• Understanding of householding algorithms and third-party data enrichment tools.

#Disneytech

The hiring range for this position in San Francisco, CA is $157,800.00 - $211,600.00 per year, in Santa Monica, CA is $144,100.00 - $193,200.00 per year and in Seattle, WA is $150,900.00 - $202,400.00 per year. The base pay actually offered will take into account internal equity and also may vary depending on the candidate’s geographic region, job-related knowledge, skills, and experience among other factors. A bonus and/or long-term incentive units may be provided as part of the compensation package, in addition to the full range of medical, financial, and/or other benefits, dependent on the level and position offered.

Disclaimer

This position is with Disney Entertainment & Sports LLC, which is part of a business we call Disney Entertainment and ESPN Product & Technology.

To learn more about Disney's guidelines on the use of AI for candidates, click here.

Disney Entertainment & Sports LLC is an equal opportunity employer. Applicants will receive consideration for employment without regard to race, color, religion, sex, national origin, sexual orientation, gender identity, disability or protected veteran status. Disney champions a business environment where ideas and decisions from all people help us grow, innovate, create the best stories and be relevant in a constantly evolving world.

DISABILITY ACCOMMODATION FOR EMPLOYMENT APPLICATIONS

The Walt Disney Company and its Affiliated Companies are Equal Employment Opportunity employers and welcome all job seekers including individuals with disabilities and veterans with disabilities. If you have a disability and believe you need a reasonable accommodation in order to search for a job opening or apply for a position, visit the Disney candidate disability accommodations FAQs. We will only respond to those requests that are related to the accessibility of the online application system due to a disability.

Having technical issues? View the FAQs for help.