Senior Data Engineer — Real-Time Data & Databricks

Nicoll Curtin; Leicester; Hybrid; GBP 95,755 - 125,644; Part time

Benefits offered by this job

Outside IR35

Modern tech stack

Friendly team

Job summary

A leading tech recruitment firm in the UK is seeking a Senior Data Engineer to enhance their data engineering capabilities. This hybrid contract role focuses on building ETL/ELT pipelines, optimizing data governance, and developing real-time data solutions using Azure technologies. The ideal candidate will have strong experience with Azure, Databricks, and proficiency in Python and Terraform. This position offers competitive pay of £600/day, fostering a collaborative work environment.

Qualifications

• Strong hands-on experience with Azure & Databricks.
• Proficiency in Python, Spark SQL, and Terraform.
• Solid understanding of data governance and real-time processing.
• Great communication skills and a collaborative mindset.

Responsibilities

• Building and optimising ETL/ELT pipelines with Databricks & Apache Spark.
• Designing scalable solutions using Azure Data Factory, Synapse & Data Lake.
• Implementing Delta Lake with ACID transactions and schema enforcement.
• Automating testing and CI/CD with Azure DevOps & Terraform.
• Driving data governance with Azure Purview & Unity Catalog.
• Developing real-time data solutions with Event Hubs & Structured Streaming.
• Managing infrastructure as code and optimising cloud costs.
• Working closely with engineers, QA, Product Owner, and stakeholders.

Skills

Azure

Databricks

Python

Spark SQL

Terraform

Data governance

Real-time processing

Communication

Collaboration

Job description

A leading tech recruitment firm in the UK is seeking a Senior Data Engineer to enhance their data engineering capabilities. This hybrid contract role focuses on building ETL/ELT pipelines, optimizing data governance, and developing real-time data solutions using Azure technologies. The ideal candidate will have strong experience with Azure, Databricks, and proficiency in Python and Terraform. This position offers competitive pay of £600/day, fostering a collaborative work environment.