Principal Data Engineer

Disney Entertainment and ESPN Product & Technology; New York (NY); Vor Ort; USD 180.000 - 241.000; Vollzeit

Zusammenfassung

Disney Entertainment and ESPN Product & Technology is seeking a Principal Data Engineer to lead Identity and Device data engineering for marketing measurement and analytics. You will shape pipeline architecture and platform patterns in collaboration with Marketing and Consumer Data teams.

You will own end-to-end delivery for Identity360 and identity datasets, establishing engineering quality standards and scalable data pipelines on Databricks, AWS, and orchestration tooling.

Qualifikationen

• 10+ years of big data engineering experience and building large data pipelines.
• Strong Python/Scala/Java programming and AWS fundamentals.
• Experience with distributed systems at scale (Hadoop, Databricks, Spark).
• Knowledge of data modeling and data warehousing best practices.

Aufgaben

• Define and drive technical strategy for Identity & Device data products in Marketing and Consumer Data.
• Own end-to-end delivery for Identity360 and related datasets and data products.
• Set engineering standards for quality, CI/CD, observability and governance.

Kenntnisse

Python

Scala

Java

AWS fundamentals

Ausbildung

Bachelor’s degree in Computer Science or related field

Tools

Databricks

Hadoop

Spark

Presto

Hive

Airflow

Terraform

CI/CD

Jobbeschreibung

Disney Entertainment and ESPN Product & Technology is hiring a Principal Data Engineer to lead Identity and Device data engineering for marketing measurement and analytics.

Responsibilities

• Define and drive technical strategy for Identity & Device data products in Marketing and Consumer Data, including pipeline architecture, platform patterns, and a modernization roadmap in partnership with onshore squad leaders and enterprise architecture.
• Own end-to-end delivery for complex, high-impact initiatives spanning Identity360, device/consumer identifier datasets, and dependent Marketing data products, including technical approach, sequencing, and engineering quality standards for other engineers.
• Act as principal subject matter expert and escalation point for identity foundational data, covering data modeling, batch and near-real-time processing, performance at scale, and production reliability.
• Participate in an on-call rotation and lead incident response for critical pipelines.
• Set and enforce engineering standards for the Identity & Device team and Marketing Data Engineering, including code quality, CI/CD, observability, and data governance aligned to GDPR, RTBF, and CCPA, along with reusable platform patterns across Databricks, AWS, and orchestration tooling.
• Mentor and develop engineers through design reviews, pairing, and coaching, building depth in big-data engineering, identity domain knowledge, and operational excellence.
• Collaborate cross-functionally with Product, QA, Operations, Privacy/Legal, and peer Marketing data squads (Data Collection, Activation, Acquisition) to align requirements, resolve cross-domain dependencies, and represent technical decisions to stakeholders.

Requirements

• 10+ years of related big data engineering experience, including modeling and building large data pipelines; strong experience with Python, Scala, or Java; strong AWS fundamentals (S3, Lambda, IAM) preferred.
• 5+ years hands-on experience with performance tuning at scale using distributed systems such as Hadoop or Databricks, with query/process tools such as Spark, Presto, and Hive at the petabyte level.
• Experience with at least one major MPP or cloud database technology: Snowflake, Redshift, BigQuery, or Databricks.
• Solid understanding of data modeling and data warehousing standard methodologies and practices.
• Operational experience in a multi-system/services production environment as a senior data engineer and point-of-contact/SME.
• Strong problem-solving with attention to detail, plus excellent analytical and communication skills.
• Experience with data integration and orchestration toolsets (e.g., Airflow) and CI/CD.
• Demonstrated ability with cloud infrastructure technologies, including Terraform.
• Strong understanding of slowly changing dimensions (SCD tables) and building scalable, performant datasets.
• Bachelor’s degree in computer science, Information Systems, Software, Electrical or Electronics Engineering, or comparable field of study, and/or equivalent work experience.

Technologies

• Python, Scala, Java
• AWS, S3, Lambda, IAM
• Hadoop, Databricks, Spark, Presto, Hive
• Snowflake, Redshift, BigQuery
• Airflow, CI/CD, Terraform
• GDPR, RTBF, CCPA

Benefits

• Bonus and/or long-term incentive units may be provided as part of the compensation package.
• Full range of medical, financial, and/or other benefits.

Location: New York, NY (onsite)

Compensation: USD 187,900 - 252,000 per year

Experience: 10+ years

PAY TRANSPARENCY: The hiring range for this position in SFO, CA is $187,900 - $252,000 per year and New York is $179,700 - $241,000 per year.

Note: This position is with Disney Entertainment & Sports LLC, which is part of a business we call Disney Entertainment and ESPN Product & Technology.