Senior Data Engineer

Mastek UK; City Of London; Hybrid; GBP 70,000 - 90,000; Full time

Job summary

A leading technology company is seeking a Senior Data Engineer to work in London. The ideal candidate should have over 10 years of experience in data engineering with deep expertise in Azure Databricks and strong programming skills in Python and Spark. This role requires a solid understanding of data warehousing and integration patterns, along with experience in a financial services context. You will work on complex data pipelines and architecture in a collaborative environment.

Qualifications

• 10+ years of experience in data engineering.
• 3+ years of hands-on experience with Azure Databricks.
• Experience working in financial services or economic data environments.

Skills

Data engineering experience

Azure Databricks

Python

Spark (PySpark) or Scala

Data warehousing principles

Data integration patterns

Azure Data Factory

Version control systems (Git)

Agile development methodologies

Data governance processes

Education

Azure Data Engineer Associate certification

Tools

Tableau

Power BI

Azure DevOps

Docker

Kubernetes

Job description

Overview

Senior Data Engineer

Location: London, UK (3 days in the office)

SC Cleared: Required

Essential Skills & Experience

• 10+ years of experience in data engineering, with at least 3+ years of hands-on experience with Azure Databricks.
• Strong proficiency in Python and Spark (PySpark) or Scala.
• Deep understanding of data warehousing principles, data modelling techniques, and data integration patterns.
• Extensive experience with Azure data services, including Azure Data Factory, Azure Blob Storage, and Azure SQL Database.
• Experience working with large datasets and complex data pipelines.
• Experience with data architecture design and data pipeline optimization.
• Proven expertise with Databricks, including hands-on implementation experience and certifications.
• Experience with SQL and NoSQL databases.
• Experience with data quality and data governance processes.
• Experience with version control systems (e.g., Git).
• Experience with Agile development methodologies.
• Excellent communication, interpersonal, and problem-solving skills.
• Experience with streaming data technologies (e.g., Kafka, Azure Event Hubs).
• Experience with data visualisation tools (e.g., Tableau, Power BI).
• Experience with DevOps tools and practices (e.g., Azure DevOps, Jenkins, Docker, Kubernetes).
• Experience working in a financial services or economic data environment.
• Azure certifications related to data engineering (e.g., Azure Data Engineer Associate).