Data Engineer for AI-Driven Data Pipelines on Azure

Capgemini Invent; Województwo pomorskie; Hybrid; PLN 180,000 - 260,000; Full time

Benefits offered by this job

Yearly bonus

Private medical care

Life insurance

Training & certifications

Language learning access

Job summary

Capgemini Invent is seeking an experienced Data Engineer to join a strategic Data & AI initiative serving a leading international legal organization. You will build scalable data pipelines, ingest data from enterprise systems and APIs, and deliver data products with Delta Lake and Databricks.

Hybrid working options are available in Poland. The role requires 5+ years of experience, strong PySpark, Python and SQL, plus expertise in Azure Databricks, Delta Lake, and CI/CD pipelines.

Qualifications

• 5+ years of Data Engineering experience.
• Strong PySpark, Python and SQL skills.
• Hands-on experience with Azure Databricks and Delta Lake.
• Experience delivering end-to-end data platforms and ingesting data from enterprise systems and APIs.
• Knowledge of CDC, SCD2, data quality and governance practices.
• Experience with Azure DevOps, CI/CD and YAML pipelines.
• Fluent English; on-site availability in Gdańsk or Wrocław.

Responsibilities

• Build and maintain scalable data pipelines using Azure Databricks and Delta Lake.
• Ingest, cleanse, transform and deliver data for analytics and AI use cases.
• Develop metadata-driven and reusable data-engineering frameworks.
• Implement data-quality, CDC, SCD2, observability and audit solutions.
• Deploy data products using Databricks Asset Bundles and Azure DevOps.
• Optimize Delta Lake models and ETL/ELT processes.
• Collaborate with architects, engineers and business stakeholders.

Skills

PySpark

Python

SQL

Delta Lake

ETL/ELT

Data governance

Analytical thinking

Tools

Azure Databricks

Azure DevOps

CI/CD

YAML pipelines

Job description

Capgemini Invent is seeking an experienced Data Engineer to join a strategic Data & AI initiative serving a leading international legal organization. You will build scalable data pipelines, ingest data from enterprise systems and APIs, and deliver data products with Delta Lake and Databricks.

Hybrid working options are available in Poland. The role requires 5+ years of experience, strong PySpark, Python and SQL, plus expertise in Azure Databricks, Delta Lake, and CI/CD pipelines.