Senior Data Engineer — Azure Databricks, Hybrid

Databricks, Inc.; Langley; Hybrid; GBP 85,000 - 100,000; Full time

Benefits offered by this job

Bonus 10%

Job summary

Databricks, Inc. is seeking a Senior Data Engineer to join its Enterprise Data function in a hybrid London/Slough setting.

The role focuses on delivering complex data engineering work on Azure Databricks and leading other engineers while maintaining hands-on technical leadership. You will design scalable data solutions, build batch and streaming pipelines, and develop trusted datasets for reporting, analytics and AI, using Python, PySpark, Delta Lake and SQL.

Qualifications

• 5–7 years of data engineering experience.
• Strong and current Azure Databricks experience.
• Advanced Python, PySpark and SQL skills.
• Strong knowledge of Delta Lake, Unity Catalog and data modelling.
• Experience with Azure Data Factory and Azure Data Lake Storage.
• Experience of infrastructure as code and Azure resource management.
• A strong understanding of software engineering principles, automated testing and CI/CD.
• Recent experience leading or mentoring Data Engineers.
• The ability to review and assure the quality of other engineers' work.
• An interest in architecture and the ambition to take on broader technical responsibility.

Responsibilities

• Deliver complex data engineering work.
• Provide day-to-day technical leadership to other engineers.
• Act as the number-two technologist within the Enterprise Data team.
• Design and build scalable data solutions using Azure and Databricks.
• Develop batch and streaming data pipelines from databases, APIs and Kafka.
• Build trusted datasets for reporting, analytics and AI.
• Work with Python, PySpark, Delta Lake, Unity Catalog and SQL.
• Develop dimensional data models using Kimball principles.
• Review code, technical designs, Terraform and testing.
• Ensure the team's engineering work meets quality and governance standards.
• Support Data Engineers with technical decisions and complex problems.
• Contribute to architecture and wider technical direction of the data function.
• Improve CI/CD, automated testing, monitoring and platform reliability.

Skills

Azure Databricks

Python

PySpark

SQL

Delta Lake

Unity Catalog

Terraform

CI/CD

Data modeling

Architecture leadership

Tools

Azure Data Factory

Azure Data Lake Storage

Terraform

Job description

Databricks, Inc. is seeking a Senior Data Engineer to join its Enterprise Data function in a hybrid London/Slough setting.

The role focuses on delivering complex data engineering work on Azure Databricks and leading other engineers while maintaining hands-on technical leadership. You will design scalable data solutions, build batch and streaming pipelines, and develop trusted datasets for reporting, analytics and AI, using Python, PySpark, Delta Lake and SQL.