Azure Databricks Lead - Data Pipelines & Delta Lake

Capgemini; Charlotte (NC); Hybrid; USD 103,330 - 128,656; Full time

Benefits offered by this job

Paid time off (12-25 days)

Medical, dental, and vision coverage

Retirement savings plans (401(k))

Job summary

Capgemini is seeking an experienced Azure Databricks Engineer based in Charlotte, NC or Jersey City, NJ. The role involves designing, developing, and optimizing ETL pipelines using Azure Databricks and working closely with cross-functional teams to ensure performance and efficiency.

Key responsibilities include building scalable data ingestion workflows, implementing data governance best practices, and collaborating with business stakeholders. The compensation range for this position is between $103,330 and $128,656, along with a comprehensive benefits package.

Qualifications

• Strong expertise in cloud-based data engineering ETL development.
• Hands-on experience with building scalable data pipelines on Azure.
• Ability to implement DevOps processes using Git and CI/CD.

Responsibilities

• Design, develop, and optimize ETL/ELT pipelines using Azure Databricks PySpark.
• Implement transformation logic and data cleansing frameworks.
• Collaborate with business stakeholders to translate requirements into technical solutions.

Skills

Azure Databricks

PySpark

Azure Data Factory

ETL Development

Data Engineering

Tools

Delta Lake

Azure SQL

Event Hub

Azure DevOps

Job description

Capgemini is seeking an experienced Azure Databricks Engineer based in Charlotte, NC or Jersey City, NJ. The role involves designing, developing, and optimizing ETL pipelines using Azure Databricks and working closely with cross-functional teams to ensure performance and efficiency.

Key responsibilities include building scalable data ingestion workflows, implementing data governance best practices, and collaborating with business stakeholders. The compensation range for this position is between $103,330 and $128,656, along with a comprehensive benefits package.