Senior Data Engineer — Azure, Databricks & Lakehouse

DCG; United Kingdom; Remote; GBP 80,000 - 120,000; Full time

Benefits offered by this job

Private medical care

Co-financing for the sports card

Dedicated consultant support

Job summary

DCG, a recruitment company, seeks a data engineer to design, develop, and maintain scalable data solutions across Azure Synapse, Azure Databricks, and Data Lakehouse environments.

You will implement ETL/ELT processes, build Databricks workflows, and collaborate with architects and stakeholders to translate business needs into technical solutions.

The role requires strong Spark, Python, SQL skills, experience with Unity Catalog, Delta Live Tables, CI/CD, and a proactive approach.

Qualifications

• Strong experience with Apache Spark, Python, and SQL.
• Proven experience in building and maintaining ETL/ELT processes.
• Practical experience with Databricks, Delta Live Tables (DLT), and Unity Catalog.
• Understanding of Lakehouse architecture, data governance, security, compliance, and operational excellence.
• Familiarity with CI/CD, Azure DevOps, infrastructure automation, monitoring, and logging.
• Strong analytical and problem-solving skills.
• Ability to work independently with a high level of ownership and accountability.
• Proactive approach to identifying risks, blockers, and improvement opportunities.
• Strong communication and stakeholder management skills across technical and business teams.
• Experience working in dynamic environments with frequently changing priorities and cross-functional teams.
• Ability to make pragmatic decisions and deliver results despite incomplete requirements.
• Adaptability, resilience, self-organization, and ability to independently drive initiatives and propose practical solutions.
• English proficiency at minimum B2+ level

Responsibilities

• Design, develop, and maintain scalable data solutions across Azure Synapse, Azure Databricks, and Data Lakehouse environments
• Implement and optimize ETL/ELT processes for ingesting, transforming, and serving data across multiple business domains
• Develop and maintain data processing solutions using Python, SQL, and Apache Spark
• Support migration and modernization initiatives, including Unity Catalog adoption, platform upgrades, and data platform transformations
• Build and maintain Databricks workflows, Delta Live Tables (DLT), metadata-driven frameworks, and orchestration processes
• Collaborate with architects, analysts, and business stakeholders to translate business requirements into technical solutions
• Ensure data quality, reliability, performance, and security across the data platform
• Troubleshoot production incidents, perform root cause analysis, and implement corrective actions
• Contribute to CI/CD, infrastructure automation, monitoring, logging, and operational best practices
• Optimize data pipelines, storage structures, and processing workloads for efficiency and cost reduction
• Participate in solution and architecture reviews while promoting engineering standards and governance principles
• Support knowledge sharing, mentoring, and technical development within the team
• Collaborate with Data Architects and platform teams to align solutions with enterprise data architecture, Lakehouse principles, and governance requirements
• Support reporting, analytics, and AI/ML initiatives by providing reliable and scalable data foundations

Skills

Apache Spark

Python

SQL

Databricks

Delta Live Tables

Unity Catalog

CI/CD

Tools

Azure Synapse

Azure Databricks

Data Lakehouse

Job description

DCG, a recruitment company, seeks a data engineer to design, develop, and maintain scalable data solutions across Azure Synapse, Azure Databricks, and Data Lakehouse environments.

You will implement ETL/ELT processes, build Databricks workflows, and collaborate with architects and stakeholders to translate business needs into technical solutions.

The role requires strong Spark, Python, SQL skills, experience with Unity Catalog, Delta Live Tables, CI/CD, and a proactive approach.