Azure Databricks Data Engineer: ETL & Data Lake Architect

Capgemini

Greater London

Hybride

GBP 80 000 - 110 000

Plein temps

14 jours+

Obtenez une réponse de cet employeur

Votre CV et votre lettre de motivation, adaptés exactement à ce que l’employeur recherche.

Passez les filtres ATS

Résumé du poste

Capgemini in London seeks a Databricks Developer to design and maintain scalable ETL/ELT pipelines on Azure Databricks. You will develop data processing solutions with PySpark, Spark SQL and Python, ingesting and optimizing data from diverse sources.

You will build data lakes with Bronze-Silver-Gold layers and integrate Databricks with ADF, ADLS Gen2, Synapse, and other cloud services while ensuring governance and security.

Qualifications

  • Hands-on experience with Azure Databricks platform and Spark-based tooling.
  • Strong SQL and Python coding skills for data engineering
  • Experience integrating with cloud services (ADF, ADLS Gen2) and data governance practices

Responsabilités

  • Design, develop and maintain scalable ETL/ELT pipelines using Azure Databricks
  • Develop data processing solutions using PySpark, Spark SQL and Python
  • Ingest, transform and optimize structured and unstructured data from multiple sources
  • Build and maintain data lakes and medallion architecture (Bronze, Silver, Gold)
  • Integrate Databricks with Azure Data Factory, ADLS Gen2, Synapse and other cloud services
  • Optimize Spark jobs for performance and cost efficiency
  • Implement data quality checks, monitoring and troubleshooting processes
  • Collaborate with business stakeholders, data analysts and architects to understand data requirements
  • Ensure adherence to data governance, security and compliance standards

Connaissances

Azure Databricks
PySpark
Spark SQL
Python
Azure Data Factory
ADLS Gen2
SQL
Git CI/CD
DevOps
Data modelling

Description du poste

Capgemini in London seeks a Databricks Developer to design and maintain scalable ETL/ELT pipelines on Azure Databricks. You will develop data processing solutions with PySpark, Spark SQL and Python, ingesting and optimizing data from diverse sources.

You will build data lakes with Bronze-Silver-Gold layers and integrate Databricks with ADF, ADLS Gen2, Synapse, and other cloud services while ensuring governance and security.

Obtenez votre examen gratuit et confidentiel de votre CV.

ou faites glisser et déposez votre fichier ici.

Similar jobs

Postes similaires à comparer

Azure Databricks Engineer for Data Pipelines & Lakes
Azure Databricks Engineer for Data Pipelines & Lakes

Capgemini • Greater London

Hybride
GBP 70 000 - 100 000
Disability Confident Employer (Level 2
Azure Databricks Lead Engineer | Data & Platform Ops
Azure Databricks Lead Engineer | Data & Platform Ops

Sogeti • Grande-Bretagne

Hybride
GBP 70 000 - 110 000
Senior Data Engineer — Azure, Databricks & Lakehouse
Senior Data Engineer — Azure, Databricks & Lakehouse

DCG (Poland) • Grande-Bretagne

À distance
GBP 80 000 - 120 000
Private medical care
Co-financing for the sports card
Dedicated consultant support
Azure Databricks Engineer — Hybrid Data Solutions
Azure Databricks Engineer — Hybrid Data Solutions

Capgemini • Greater London

Hybride
GBP 70 000 - 110 000
Azure Databricks Architect — Senior Cloud Data Leader
Azure Databricks Architect — Senior Cloud Data Leader

Sogeti • Greater London

Hybride
GBP 90 000 - 130 000
Azure Databricks Platform Architect
Azure Databricks Platform Architect

Sogeti • Grande-Bretagne

Hybride
GBP 90 000 - 130 000
Hybrid work model
Training & development opportunities
Data Engineer - Azure Databricks Data Pipelines
Data Engineer - Azure Databricks Data Pipelines

SSE Enterprise • Reading

Hybride
GBP 36 000 - 54 000
Private healthcare discounts
Gym memberships
Online GP service
+3
Remote Data Engineer: Azure Databricks & Data Pipelines
Remote Data Engineer: Azure Databricks & Data Pipelines

MBN Solutions • Grande-Bretagne

À distance
GBP 70 000 - 90 000
Azure Databricks Data Engineer | Remote & Secure Cloud
Azure Databricks Data Engineer | Remote & Secure Cloud

Atos • Wideopen

À distance
GBP 70 000 - 110 000
25 days leave + holidays
Pension scheme
Private medical cover
+3
Senior Azure Data Engineer (Databricks) - Real-Time Pipelines
Senior Azure Data Engineer (Databricks) - Real-Time Pipelines

Capco • City of Edinburgh

Hybride
GBP 70 000 - 100 000
Discretionary bonus
Health insurance
Pension
+3