Senior Azure Databricks Data Engineer & Tech Lead

AgileGrid Solutions; United Kingdom; On-site; GBP 90,000 - 120,000; Full time

Benefits offered by this job

Health benefits

Pension schemes

Paid leave

Professional development

Job summary

AgileGrid Solutions seeks a Senior Data Engineer to join our Enterprise Data team. This hands-on role combines technical excellence with leadership to guide other engineers in designing, developing, and maintaining scalable Azure-based data solutions centered on Databricks.

You will mentor engineers, drive architectural decisions, and ensure high-quality delivery through CI/CD and automated testing. The ideal candidate has 5–7 years of cloud data experience, Azure Databricks, Python, PySpark,

Qualifications

• 5–7 years of data engineering experience.
• Cloud-based data platforms, especially Azure.
• Experience with Azure Databricks is essential.
• Strong Python, PySpark and SQL skills.
• Deep understanding of Delta Lake, Unity Catalog and data modeling.

Responsibilities

• Design and develop scalable data solutions using Azure and Databricks.
• Build and optimize batch and streaming data pipelines from databases, APIs, and Kafka.
• Create trusted datasets for reporting, analytics, data science, and AI initiatives.
• Write and optimize Python, PySpark, Delta Lake, Unity Catalog, and SQL code.
• Review code, Terraform configurations, and testing processes to ensure quality and standards.

Skills

Azure Databricks

Python

PySpark

SQL

Delta Lake

Unity Catalog

Azure Data Factory

Azure Data Lake Storage

Terraform

CI/CD

Testing

Mentorship

Tools

Terraform

Azure Databricks

Azure Data Factory

Azure Data Lake Storage

Job description

AgileGrid Solutions seeks a Senior Data Engineer to join our Enterprise Data team. This hands-on role combines technical excellence with leadership to guide other engineers in designing, developing, and maintaining scalable Azure-based data solutions centered on Databricks.

You will mentor engineers, drive architectural decisions, and ensure high-quality delivery through CI/CD and automated testing. The ideal candidate has 5–7 years of cloud data experience, Azure Databricks, Python, PySpark,