Data Engineer (SC Cleared)

Scrumconnect Consulting; London; On-site; GBP 55,000 - 75,000; Full time

Job summary

A leading digital consultancy in the UK is seeking an experienced Data Engineer to develop and maintain data products on a Strategic Data Platform. You will work within Agile teams, focusing on designing scalable data architectures, primarily utilizing Azure tools. Candidates must have active SC clearance and proven experience with Azure Data Factory, Power BI, and Python, among other key skills.

Qualifications

• Active SC Clearance must be valid and in place at application stage.
• Proven experience with Azure Data Factory and Azure Synapse.
• Experience in Azure DevOps and Microsoft Azure.
• Strong proficiency in Power BI, including semantic models.
• Proficiency in Python including PySpark.
• Proficient in SQL and dimension data modelling.
• Experience with Terraform for infrastructure-as-code deployments.

Responsibilities

• Design, build, and maintain data solutions using Azure tools.
• Manage data development lifecycle within Agile teams.
• Integrate and automate data workflows into Azure DevOps pipelines.
• Create and maintain dimension data and semantic models for Power BI.
• Develop complex dashboards, visualisations, and reporting solutions.
• Implement data governance and quality checks.
• Migrate legacy data abilities to modern Azure platforms.
• Collaborate with stakeholders for technical solutions.
• Coach and mentor team members in best practices.

Skills

Azure Data Factory

Azure Synapse

Azure DevOps

Power BI

Python

SQL

Dimension data modelling

Terraform

Agile environments

Business analysis

Job description

Job Description

About Scrumconnect Consulting:

Scrumconnect Consulting is a multi-award-winning digital consultancy, recognised for delivering impactful technology solutions across UK government departments. Our work has positively influenced the lives of over 40 million UK citizens. With a strong commitment to user-centred design and agile delivery, and more to deliver innovative digital services that matter

Clearance:Active SC clearance is mandatory

Role Overview

We are looking for an experienced Data Engineer to develop and maintain data products and a Strategic Data Platform. You will work as part of multi-functional Agile delivery teams, ensuring operational stability, ongoing support, and enhancement of data solutions. The role focuses on designing, delivering, and optimising scalable data architectures, with a strong emphasis on Azure-based tools and modern engineering practices.

Key Responsibilities

• Design, build, and maintain data solutions usingAzure Data FactoryandAzure Synapse.
• Manage data development lifecycle within Agile delivery teams.
• Integrate and automate data workflows intoAzure DevOpspipelines.
• Create and maintaindimension data modelsandsemantic modelsfor integration withPower BI.
• Develop complex dashboards, visualisations, and reporting solutions usingPower BI.
• Implement data governance, quality checks, and profiling to ensure accuracy and compliance.
• Migrate legacy data capabilities to modern Azure-based platforms.
• Collaborate with business stakeholders to translate requirements into robust technical solutions.
• Coach and mentor team members to enhance capability in data engineering best practices

Essential Skills & Experience

• Active SC Clearance(must be valid and in place at application stage).
• Proven experience with:
• Azure Data Factory & Azure Synapse
• Azure DevOps&Microsoft Azure
• Power BI(including semantic models)
• Python(includingPySpark)
• SQL(proficient level)
• Dimension data modelling
• Experience withTerraformfor infrastructure-as-code deployments.
• Ability to deliver solutions using structured and unstructured data.
• Experience inAgileenvironments.
• Strong business analysis skills to understand service needs and document accurately.
• Proven experience in legacy migration projects within complex organisations.
• Ability to work collaboratively and communicate effectively with technical and non-technical stakeholders.

Desirable

• Azure certifications (Data Engineering, Data Scientist, or related specialisation).
• Knowledge of GDPR compliance, data security, and governance best practices.