Lead Data Engineer

The Walt Disney Company; City Of London; On-site; GBP 70,000 - 90,000; Full time

Job summary

A leading entertainment firm in the UK is seeking a Principal Data Platforms Engineer to design and maintain data solutions across EMEA. This role requires expertise in Snowflake and AWS, with responsibilities including enhancing key data pipelines and providing mentorship. The ideal candidate should have strong programming skills and a commitment to innovation and team collaboration. Competitive compensation and growth opportunities are available.

Qualifications

• Experience with cloud-based data warehouses essential.
• Proven track record delivering reliable data solutions.
• Strong proficiency in Python and Django for web applications.

Responsibilities

• Own and enhance EMEA's Content Operations data pipelines.
• Maintain and optimize Snowflake environments.
• Lead the development of Retail Analytical databases.

Skills

Cloud-based data warehouses

Amazon Web Services (AWS)

Snowflake expertise

Python

Django

CI/CD processes

SQL

DataOps.LIVE

Strong communication skills

Tools

Snowflake

DBT

Docker

Informatica

Job description

The Principal Data Platforms Engineer plays a critical role in designing, implementing, and maintaining robust data solutions that enable scalable businesses operations across the EMEA region. This role combines deep technical expertise with strategic ownership to drive and develop and maintain end-to-end data pipelines that enable access to high-quality data for informed business decision‑making and operational efficiency. This role will own and enhance key data pipelines powering across many lines of business (LOBs), such as Media Networks and DTC, ensuring stability and performance, while continuing exploring new innovative methods leveraging our preferred technical platforms such as Snowflake, AWS, and DataOps.LIVE. Furthermore, the Principal Data Platforms Engineer will act as bridge across the team, ensuring data platforms best practices are adopted and collaboration with other technical leaders across the company to align and innovate on shared data solutions. Lastly, this role will act as a mentor, fostering a culture of knowledge sharing, constructive feedback, and high standards through peer reviews, coding excellence, and solution design‑playing a key role in the overall team development.

• Own and enhance EMEA's Content Operations pipelines, ensuring stability, performance, and alignment with evolving the business needs. The role particularly focused on the data sets that are powering the Planning Module and the Vendorbase applications, which includes our ordering tool and related supply chain applications
• Maintain and optimise the team's Snowflake environments, applying hotfixes, monitoring performance, and planning migrations when and where needed. This also covers all framework used by the team outside of Snowflake, including DBT, Terraform and Django.
• Lead and maintain the development of Disney's Retail Analytical databases enabling analysts to provide actionable insights to the business teams. This includes the continuous support and integration of the Amazon product catalogues across EMEA, the integration of the retail activity data sets
• Software / Application Development
• Design and implement robust, scalable, and cost‑effective software solutions that meet business requirements, leveraging tools like Snowflake, AWS, and DataOps.LIVE
• Continue the build out and support of the Data Platforms Django‑based applications leveraged by the Content Operations and business teams. This includes our deal management tool and our content planning application and any future applications.
• Continue to support the suite of financial applications SII, SDI and EBILL through the administration of the Informatica platform and affiliated, on‑premises infrastructures.
• Support in the alignment of cross‑team technical design principles and technology choices for applications developed outside of Data Platforms
• Maintain the MDS application used to harmonise metadata across all pipelines) and its future development
• Continue to own, develop, and distribute the Data Platforms' internal Python library that enables a unified approach for interfacing with Disney's preferred platforms such as Snowflake, AWS, Globalscape, and encourage adoption by the wider team
• Application Monitoring
• Implement a rigorous and consistent monitoring approach to ensure that our applications and data pipelines are operational. And if an issue does occur, that our monitoring tools promptly alert the team for issue remediation and minimal disruption to business continuity.
• Ongoing enhancement of enhancement of our orchestrator's infrastructure (DataOps.LIVE) and ensure all Docker images are well defined and meeting the team's requirements-driving development efficiency.
• Team Guidance and Mentorship
• Act as a key technical expert within the department, offering guidance on the usage of Snowflake, AWS resources provisioning, and other Disney‑preferred development tools
• Provide guidance in the development of the EMEA CRM databases-supporting the Senior Data Engineer-and ensuring best practices are applied and technical decisions are challenged
• Provide mentorship and guidance to junior team members, fostering a culture of knowledge sharing, peer reviews, and overall technical excellence
• Drive internal documentation on Confluence and initiate training initiatives, ensuring knowledge is shared amongst the team and readily accessible
• Innovation and Driving Change
• Champion innovation and efficiency, identifying opportunities for process enhancements to improve team workflows and overall productivity
  Experience with cloud-based data warehouses
• Experience with Amazon Web Services (AWS), including resource provisioning and management
• Proven track record of delivering business‑critical data solutions with a focus on reliability and long‑term sustainability
• Direct experience in building web‑based software, ideally using Django
• Experience with DBT for data transformation and manipulation
• Experience with DataOps.LIVE or a similar orchestration platform
• Previous experience in the broadcast/media industry is desirable
• Expertise in best practice design, development, and management of Snowflake data pipelines, ensuring scalability, performance, and cost‑effectiveness
• A strong proficiency in Python and Django, and JavaScript for web‑based applications and integrations
• Strong experience with CI/CD processes and version control tools to enable automated and reliable pipeline deployments
• Proficiency with SQL, XML, JSON, and web services development
• Strong data modelling skills for optimized data warehouse performance
• Exceptional communication skills (written and oral) with the ability to articulate complex technical concepts to both technical and non‑technical stakeholders
• Demonstrable presentation and influencing skills to drive adoption of best practices across teams and projects
• Commitment to ongoing professional development, staying current with emerging technologies and best practices to deliver innovative solutions that minimize operational overhead
• Advanced knowledge of GIT version control and associated workflows