Senior Data Engineer – Databricks

Career Choices Dewis Gyrfa Ltd; City Of London; On-site; GBP 70,000 - 110,000; Full time

Benefits offered by this job

Sponsorship may be available

Job summary

7Dxperts in the UK is seeking a Senior Data Engineer to design and optimize scalable data pipelines using Databricks, PySpark, Python and SQL, gaining hands-on experience with Delta Lake and lakehouse architectures.

You will build ETL/ELT workflows, monitor performance, mentor juniors and contribute to cloud data platforms across AWS, Azure or GCP, with a focus on reliability, quality data and impactful solutions.

Qualifications

• 5+ years’ Data Engineering experience.
• Strong hands-on Databricks, PySpark, Python and SQL skills.
• Experience building and optimisation of ETL/ELT data pipelines.
• Knowledge of Delta Lake and data warehousing.
• Experience with AWS, Azure or GCP.

Responsibilities

• Build scalable pipelines using Databricks, Spark and PySpark.
• Develop ETL/ELT and data transformation workflows using Python and SQL. Create and maintain Databricks notebooks, jobs, workflows and clusters.
• Work with Delta Lake and lakehouse architectures.
• Monitor, troubleshoot and optimise pipeline performance.
• Maintain high standards of data quality and reliability.
• Automate processes and create reusable engineering solutions.
• Support and mentor junior Data Engineers.

Skills

Databricks

PySpark

Python

SQL

ETL/ELT pipelines

Delta Lake

Cloud platforms (AWS/Azure/GCP)

CI/CD

Terraform

Databricks certification

Education

Computer Science or related technical background

Job description

Databricks | Competitive Salary | UK (Remote)

Are you an experienced Data Engineer who enjoys turning complex data into reliable, scalable solutions? Do you have strong hands‑on experience with Databricks, PySpark, Python and SQL? The role 7Dxperts is looking for a Senior Data Engineer to join our UK team.

Key Responsibilities

• Build scalable pipelines using Databricks, Spark and PySpark.
• Develop ETL/ELT and data transformation workflows using Python and SQL. Create and maintain Databricks notebooks, jobs, workflows and clusters.
• Work with Delta Lake and lakehouse architectures.
• Monitor, troubleshoot and optimise pipeline performance.
• Maintain high standards of data quality and reliability.
• Automate processes and create reusable engineering solutions.
• Support and mentor junior Data Engineers.

Requirements / Qualifications

• 5+ years’ Data Engineering experience.
• Strong hands‑on Databricks, PySpark, Python and SQL skills.
• Experience building and optimising ETL/ELT data pipelines.
• Knowledge of Delta Lake and data warehousing.
• Experience with at least one of AWS, Azure or GCP. Strong troubleshooting and problem-solving skills.
• A Computer Science or related technical background.
• Experience with Unity Catalog, MLflow, CI/CD, Terraform or Databricks certification would be a bonus.

About 7Dxperts

7Dxperts helps organisations solve complex challenges using Data, Spatial and Machine Learning technologies.

We focus on building practical, targeted solutions that make a measurable difference.

What we offer

• Databricks training and upskilling.
• Support towards technical certifications.
• Exposure to AWS, Azure and GCP. Varied data engineering projects.
• Technical and leadership development.
• Sponsorship may be available depending on circumstances.