Data Engineer

SCC; United Kingdom; Hybrid; GBP 62,000 - 90,000; Full time

Job summary

SCC is looking for an experienced Data Engineer to design, build and optimise scalable data pipelines across hybrid cloud environments in London, Bracknell, Birmingham, Leeds, Manchester or Livingston.

You will work with Azure/AWS/GCP, Databricks, Snowflake and modern data platforms to support analytics, BI and data governance, while ensuring security and scalability in enterprise settings.

Qualifications

• Proven experience building scalable data pipelines in large enterprise environments.
• Experience with cloud data platforms (Azure, AWS, GCP).
• Strong SQL with large volumes of structured and unstructured data.
• Experience with ETL/ELT, data modelling and data architecture principles.
• Familiarity with CI/CD and DevOps practices.

Responsibilities

• Design, build and maintain scalable data pipelines.
• Develop and optimise ETL/ELT processes.
• Build and manage data lakes and data warehouses to support reporting.
• Collaborate with architects, analysts, data scientists and stakeholders to deliver data solutions.
• Design and maintain cloud-based and hybrid data environments, ensuring performance and security.
• Automate data workflows and deployment processes using DevOps practices where applicable.
• Monitor, troubleshoot and optimise data pipelines and platform performance.
• Provide technical leadership and best-practice guidance across data technologies.

Skills

Data pipelines

ETL/ELT

Cloud platforms (Azure/AWS/GCP)

SQL

Python/Scala/Spark

CI/CD / DevOps practices

Tools

Azure Data Factory

Databricks

Snowflake

BigQuery

Synapse Analytics

Job description

Data Engineer - London / Bracknell / Birmingham / Leeds / Manchester / Livingston - Hybrid - SCC Flex Contract

We are seeking an experienced Data Engineer to support high-profile transformation and technology initiatives across a complex enterprise environment. This is an exciting opportunity for a Data Engineer to join a dynamic client programme, driving the design, development and optimisation of modern data platforms and pipelines. The successful Data Engineer will play a key role in enabling data-driven decision-making, supporting analytics, AI and business intelligence capabilities across the organisation.

Your responsibilities as the Data Engineer:

• Design, build and maintain scalable data pipelines and integration solutions across enterprise environments.
• Develop and optimise ETL/ELT processes to support data ingestion, transformation and delivery requirements.
• Build and manage modern data platforms, data lakes and data warehouses to support business and operational reporting.
• Collaborate with architects, developers, analysts, data scientists and business stakeholders to deliver robust data solutions.
• Design and maintain cloud-based and hybrid data environments, ensuring performance, reliability and scalability.
• Support transformation programmes and continuous improvement initiatives through innovative data engineering practices.
• Ensure all data solutions align with organisational security, governance, compliance and data quality standards.
• Automate data workflows and deployment processes using DevOps and Infrastructure as Code best practices where applicable.
• Monitor, troubleshoot and optimise data pipelines, processing workloads and platform performance.
• Provide technical leadership, expertise and best-practice guidance across data engineering technologies and methodologies.
• Contribute to technical roadmaps, architecture reviews and strategic data initiatives.

As a successful Data Engineer, you will have:

• Proven experience working as a Data Engineer within large-scale enterprise environments.
• Strong experience designing, developing and supporting scalable data pipelines and data integration solutions.
• Expertise in cloud data platforms such as Microsoft Azure, AWS and/or Google Cloud Platform.
• Strong SQL development skills and experience working with large volumes of structured and unstructured data.
• Experience with modern data technologies such as Azure Data Factory, Synapse Analytics, Databricks, Snowflake, Redshift or BigQuery.
• Experience building and maintaining data lakes, data warehouses and analytics platforms.
• Strong knowledge of ETL/ELT methodologies, data modelling and data architecture principles.
• Experience with Python, Spark, Scala and/or other relevant data engineering technologies.
• Familiarity with CI/CD pipelines, DevOps methodologies and automation practices.
• Understanding of data governance, data quality, security and regulatory compliance requirements.
• Strong stakeholder management, communication and collaboration skills.
• Excellent analytical, troubleshooting and problem-solving abilities.
• Experience delivering enterprise-scale technology solutions within complex environments.
• Relevant certifications and industry experience are highly desirable.

Professional references required.

NOTE: At SCC, we take the privacy and security of your information very seriously. Any information we hold will be handled in accordance with current data protection legislation. Upon submitting your application, SCC will process your information in line with our privacy policy, which can be found on our website under Legal Privacy Notice Flexible Resourcing.