Principal Data Engineer: Azure Lakehouse Migration Lead

Allianz Technology; Minneapolis (MN); On-site; USD 110,000 - 150,000; Full time

Benefits offered by this job

Hybrid work model

Health insurance

Company bonus scheme

401(K) with company match

Paid holidays

Paid time off

Tuition reimbursement

Paid parental leave

Employee shares program

Job summary

Allianz Technology is seeking a Principal Data Engineer to hands-on re-engineer an enterprise data platform on Azure Synapse and migrate workloads to Databricks. You will reverse‑engineer existing implementations, validate data quality, and drive a migration strategy that preserves governance and control semantics.

Strong experience with lakehouse patterns, Delta Lake, and Databricks is required, along with robust Python/SQL skills and a track record in large-scale regulated environments.

Qualifications

• 7+ years of hands-on Data Engineering experience.
• Experience migrating or re-engineering enterprise data platforms.
• Deep knowledge of enterprise data governance patterns: audit trails, reconciliation, data quality controls, SCD versioning.
• Strong analytical mindset; ability to read implementations and distinguish intent from defects.
• Primarily hands-on, with ability to contribute to technical design discussions.
• Strong communication skills to engage business, governance, and engineering stakeholders.
• Experience in regulated or enterprise-scale environments (financial services a plus).

Responsibilities

• Audit, document, and validate pipeline logic, data models, transformation frameworks, and governance controls.
• Identify gaps, defects, and technical debt across the platform and remediate sub-optimal implementations.
• Ensure correctness of data processing patterns including CDC, SCD, deduplication, and reconciliation.
• Implement target-state lakehouse designs with parity and data fidelity during transitions.
• Migrate workloads to Databricks while preserving governance and control semantics.
• Re-implement ingestion, transformation, and orchestration pipelines on target platforms.
• Collaborate with business, governance, and architecture stakeholders on data rules and quality.
• Mentor engineers, review code/designs, and support decommissioning of legacy components.

Skills

Azure Synapse

Databricks

PySpark

Spark SQL

SQL

Python

Delta Lake

SCD Type 0/1/2

CDC patterns

Data governance

Data modeling

ETL/ELT

Lakehouse

Audit logging

Tools

Azure DevOps

GitHub Actions

Terraform

ARM

Docker

Kubernetes

Power BI

Job description

Allianz Technology is seeking a Principal Data Engineer to hands-on re-engineer an enterprise data platform on Azure Synapse and migrate workloads to Databricks. You will reverse‑engineer existing implementations, validate data quality, and drive a migration strategy that preserves governance and control semantics.

Strong experience with lakehouse patterns, Delta Lake, and Databricks is required, along with robust Python/SQL skills and a track record in large-scale regulated environments.