Senior Databricks Data Engineer — Spark & Azure Pipelines

Capgemini; Calgary; On-site; CAD 84,000 - 135,000; Full time

Benefits offered by this job

Vacation: 12-25 days

Medical, dental, and vision coverage

RRSP/retirement plan

Job summary

Capgemini is seeking a Databricks Data Engineer to design, develop, and maintain scalable data pipelines and cloud-based platforms. You will work with Databricks, PySpark, Python, SQL, and Delta Lake to support analytics, reporting, and ML workloads in a Calgary-based environment.

The role emphasizes building fault-tolerant pipelines, collaborating with data teams, and optimizing workloads for performance and cost.

Qualifications

• Experience in data engineering for batch and real-time pipelines.
• Strong knowledge of Delta Lake, Databricks, PySpark.
• Experience with cloud platforms (Azure or AWS).

Responsibilities

• Design, build, and optimize scalable ETL/ELT pipelines.
• Ingest and model data for analytics and ML workloads.
• Collaborate with data scientists and engineers.
• Tune Spark jobs for performance and cost efficiency.
• Implement data governance and security practices.

Skills

Spark (PySpark)

Python

SQL

Delta Lake

Tools

Databricks

Airflow

Azure Data Factory

Databricks Workflows

Job description

Capgemini is seeking a Databricks Data Engineer to design, develop, and maintain scalable data pipelines and cloud-based platforms. You will work with Databricks, PySpark, Python, SQL, and Delta Lake to support analytics, reporting, and ML workloads in a Calgary-based environment.

The role emphasizes building fault-tolerant pipelines, collaborating with data teams, and optimizing workloads for performance and cost.