Senior Data Engineer

Michael Page Technology; Greater London; On-site; GBP 161,000 - 207,000; Full time

Job summary

Michael Page Technology seeks a Senior Data Engineer to design and deliver scalable, cloud-based data solutions for business-critical decision-making in Greater London. You will build robust data pipelines, optimise performance, and drive best practices within a growing data function.

Responsibilities include ETL/ELT development, lakehouse architectures, data governance, and collaboration with analysts and data scientists.

Qualifications

• 10+ years of experience in Data Engineering.
• Strong commercial experience with Python.
• Advanced Databricks knowledge including notebooks, workflows, jobs and Delta Lake.
• Strong SQL and data modelling expertise.
• Experience in AWS, Azure, or GCP environments.
• Exposure to Snowflake, BigQuery, or Redshift is a plus.
• Familiarity with orchestration tools such as Airflow or Databricks Workflows.
• Excellent communication with technical and non-technical stakeholders.
• Strong problem-solving mindset and delivery of high-quality data solutions.

Responsibilities

• Design, develop, and maintain scalable data pipelines using Python and Databricks.
• Build and optimise ETL/ELT workflows using Spark and Delta Lake.
• Develop and maintain data models and lakehouse architectures.
• Ensure data quality, reliability, governance, and security.
• Optimise performance of distributed data processing workloads.
• Collaborate with analysts, engineers, and data scientists to support analytics.
• Implement monitoring, logging, and alerting across data platforms.
• Mentor junior engineers and promote engineering best practices.
• Contribute to evolution of the data platform and architecture.

Skills

Python

Databricks

Spark/PySpark

SQL

Data modelling

Cloud (AWS/Azure/GCP)

ETL pipelines

Airflow

Databricks Workflows

Communication

Tools

Delta Lake

Airflow

Databricks Workflows

Job description

We're seeking a Senior Data Engineer to play a key role in designing and delivering scalable, cloud-based data solutions that support business-critical decision-making. You'll work across the full data lifecycle, building robust pipelines, optimising performance, and helping drive best practice within a growing data function.

Client Details

Our client is a forward-thinking organisation undergoing significant investment in its data and technology capabilities. Operating in a fast-paced environment, they are leveraging modern cloud technologies to transform how data is used across the business and create a more data-driven culture.

Description

• • Design, develop, and maintain scalable data pipelines using Python and Databricks.
  • Build and optimise ETL/ELT workflows using Spark and Delta Lake.
  • Develop and maintain data models and lakehouse architectures.
  • Ensure high levels of data quality, reliability, governance, and security.
  • Optimise the performance of distributed data processing workloads.
  • Collaborate with analysts, engineers, and data scientists to support reporting and advanced analytics initiatives.
  • Implement monitoring, logging, and alerting frameworks across data platforms.
  • Support and mentor junior engineers while promoting engineering best practices.
  • £700-900 Day Rate (Inside IR35)
  • Contribute to the continual evolution of the organisation's data platform and architecture.

Profile

• • 10+ years' experience in Data Engineering.
  • Strong commercial experience with Python.
  • Advanced hands-on knowledge of Databricks, including notebooks, workflows, jobs, and Delta Lake.
  • Strong experience with Apache Spark / PySpark.
  • Proven track record building and maintaining scalable ETL pipelines.
  • Strong SQL and data modelling expertise.
  • Experience working within AWS, Azure, or GCP environments.
  • Exposure to modern data warehousing platforms such as Snowflake, BigQuery, or Redshift.
  • Familiarity with orchestration tools including Airflow or Databricks Workflows.
  • Excellent communication skills with the ability to engage technical and non-technical stakeholders.
  • Strong problem-solving mindset and a passion for delivering high-quality data solutions.

Job Offer

• • Opportunity to influence the direction of a modern data platform.
  • Exposure to the latest cloud, data engineering, and lakehouse technologies.
  • Collaborative and highly skilled technical environment.
  • Clear opportunity to mentor others and shape engineering best practice.
  • Flexible working arrangements.