Senior Data Engineer - Identity and Data Platforms

ESPN; Santa Monica (CA); On-site; USD 144,000 - 193,000; Full time

Job summary

Disney Entertainment & ESPN Product & Technology is seeking a senior big data engineer to design and maintain Identity Householding Data products and scalable data pipelines. This role focuses on real-time and batch data delivery and requires deep knowledge of distributed systems and cloud data tooling.

You will partner with cross-functional teams to evolve data models, improve performance, and reduce costs while practicing Agile methodologies and on-call support.

Qualifications

• Bachelor's degree in Computer science, Information Systems, Software, Electrical or Electronics Engineering, or comparable field of study, and/or equivalent work experience.
• 5+ years of related big data engineering experience modeling and developing large data pipelines.
• Hands-on experience with distributed systems such as Databricks compute using tools/libraries such as Spark, Presto, Hive to query and process large datasets at the petabyte level.
• Strong Python, Scala/Spark and SQL skills processing big datasets.
• Experience with at least one major MPP or cloud database technology (Snowflake, Redshift, Big Query, Databricks).
• Familiarity with Data Modeling techniques and Data Warehousing standard methodologies and practices.
• Operational experience in multi-system/services production environment as point-of-contact / SME.
• Problem solving skills with strong attention to detail and excellent analytical and communication skills.
• Solid experience with data integration and orchestration toolsets (e.g. Airflow), CI/CD.
• Solid experience with AWS S3.

Responsibilities

• Build and maintain product-driven initiatives to improve and expand Identity and Device data product offerings.
• Enhance/improve the underlying codebase to be extensible, reusable, and maintainable. Increase unit test coverage on the existing codebase.
• Enhance/redesign existing data models with input from Architecture in order to modernize products and/or improve performance and efficiency, and decrease costs.
• Promote and support Agile methodologies such as Scrum, Kanban, and Scrumban by actively participating in regular ceremonies such as stand-up, retrospectives and sprint planning.
• Collaborate with your squad, Product Managers, Designers, QA, Operations, and other stakeholders to understand requirements and articulate technical decisions and outcomes.
• Participate in on-call support
• Participate in code reviews and problem-solving.

Skills

Python

Scala/Spark

SQL

Distributed systems

Education

Bachelor's degree in Computer Science or related field

Tools

Databricks

Spark

Presto

Hive

Airflow

AWS S3

Job description

Disney Entertainment & ESPN Product & Technology is seeking a senior big data engineer to design and maintain Identity Householding Data products and scalable data pipelines. This role focuses on real-time and batch data delivery and requires deep knowledge of distributed systems and cloud data tooling.

You will partner with cross-functional teams to evolve data models, improve performance, and reduce costs while practicing Agile methodologies and on-call support.