Data Engineer

Department for Culture, Media and Sport; Manchester; On-site; GBP 50,000 - 70,000; Full time

Job summary

Department for Culture, Media and Sport in Manchester is seeking an SEO Data Engineer / Data Engineer to translate user requirements into production-grade data architectures. You will build and maintain data pipelines on the Databricks platform, ensuring secure, reliable access to data for analytics and BI.

You will work in an agile multidisciplinary team, collaborating with data analysts and business stakeholders to deliver value and maintain data quality and governance.

Responsibilities

• Write clean, efficient, reusable and tested code with version control and CI/CD automation.
• Design and maintain robust ETL/ELT pipelines and data flows using Databricks.
• Design and maintain structured data models for conformant, query-friendly data.
• Monitor, troubleshoot and optimise pipeline performance and system reliability.
• Collaborate with data analysts, data scientists and stakeholders in agile delivery.
• Apply data classification, security standards and governance to manage access and metadata.

Skills

Databricks

ETL/ELT pipelines

CI/CD automation

Data modelling

Data governance

Security standards

Cross-functional collaboration

Tools

Databricks

Job description

We are transforming DCMS to meet the demands of an agile, delivery-focused department leading on some of the highest priority work for the government's agenda to drive growth, enrich lives and level-up society. Our modernisation plans include critical investment in our systems, ensuring they are fit for purpose and reducing the manual input required, as well as the ongoing development of a new central function to transform the way DCMS uses technology, data and digital services. We also want to transform how we solve policy challenges by using user centred design and collaboration with using multi-disciplinary teams. This is an incredibly exciting time to join the team within DCMS. Our mission is to digitally enable how DCMS delivers its priority outcomes which can adapt and respond to the changing needs of the sectors and partners we work with. Can you help us get there? You will work within an agile multidisciplinary team supporting the design, development and maintenance of data pipelines and systems on our Databricks platform. You will help ensure that data within the department is accessible, reliable and secure, enabling data-driven decision making. As an SEO Data Engineer, you will work with autonomy to translate user requirements and operational needs into production-ready data architectures while supporting best practices across the team. As a Data Engineer, you will:

• Write clean, efficient, reusable and tested code, applying best practices, version control and CI/CD automation.
• Design, implement and maintain robust ETL/ELT pipelines and data flows connecting operational systems to analytics and Business Intelligence (BI) platforms using Databricks.
• Design and maintain structured data models to ensure data is conformed, efficient to query and business-ready.
• Monitor, troubleshoot and optimise pipeline performance and system reliability, investigating root causes and putting preventative measures in place.
• Collaborate with data analysts, data scientists and business stakeholders to gather requirements and deliver business value through iterative agile delivery.
• Apply data classification, security standards, and governance procedures to manage access controls, maintain metadata and ensure regulatory compliance.

We are committed to supporting professional development and will provide opportunities for learning, mentoring and skills development throughout the appointment.