Senior Data Engineer & Tech Lead - Azure Databricks (Hybrid)

Databricks, Inc.; Greater London; Hybrid; GBP 85,000 - 100,000; Full time

Benefits offered by this job

10% Bonus

Job summary

Databricks, Inc. in London (hybrid) is seeking a Senior Data Engineer to lead complex data initiatives within the Enterprise Data function. The role centers on Azure Databricks, Python, PySpark and SQL, delivering scalable data solutions and trusted datasets for analytics and AI.

You will provide day-to-day technical leadership, review designs and mentor other engineers, while driving best practices in CI/CD, testing and platform reliability.

Qualifications

• 5+ years of data engineering experience.
• Strong Azure Databricks experience.
• Advanced Python, PySpark and SQL skills.
• Knowledge of Delta Lake and Unity Catalog.
• Experience with Azure Data Factory and Azure Data Lake Storage.
• Infrastructure as code with Azure resources (Terraform).
• Solid software engineering practices: automated testing and CI/CD.
• Experience mentoring data engineers and leading teams.

Responsibilities

• Deliver complex data engineering work.
• Provide day-to-day technical leadership to other engineers.
• Act as the number-two technologist in the Enterprise Data team.
• Design and build scalable data solutions using Azure and Databricks.
• Develop batch and streaming data pipelines from databases, APIs and Kafka.
• Build trusted datasets for reporting, analytics and AI.
• Review code and technical designs; ensure quality and governance.

Skills

Azure Databricks

Python

PySpark

SQL

Delta Lake

Unity Catalog

Data Modelling

Azure Data Factory

Azure Data Lake Storage

Terraform

CI/CD

Automated Testing

Leadership

Tools

Databricks

Terraform

Azure Data Factory

Job description

Databricks, Inc. in London (hybrid) is seeking a Senior Data Engineer to lead complex data initiatives within the Enterprise Data function. The role centers on Azure Databricks, Python, PySpark and SQL, delivering scalable data solutions and trusted datasets for analytics and AI.

You will provide day-to-day technical leadership, review designs and mentor other engineers, while driving best practices in CI/CD, testing and platform reliability.