Principal Data Engineer: Azure Lakehouse Migration & Governance

Allianz; Minneapolis (MN); Hybrid; USD 110,000 - 150,000; Full time

Benefits offered by this job

Hybrid work model

Health insurance

401(k) with company match

Paid holidays

Job summary

Allianz Technology seeks a Principal Data Engineer to hands-on re-engineer an enterprise data platform on Azure Synapse, with migration to Databricks. You will audit, reverse-engineer, and validate end-to-end data flows from ingestion to consumption, implementing lakehouse designs and ensuring governance and data quality.

The role involves parallel-run validation, complex data pipelines, and mentoring junior engineers.

Qualifications

• 7+ years of hands-on experience in Data Engineering, including platform migration or re-engineering work.
• Proven track record with enterprise data platforms and governance patterns.
• Strong analytical mindset to read implementations and distinguish intent from defects.

Responsibilities

• Contribute to the technical assessment and re-engineering of an existing enterprise data platform.
• Reverse-engineer, document, and validate pipeline logic, data models, and governance controls.
• Identify gaps and technical debt and remediate sub-optimal implementations.
• Ensure data processing patterns correctness (SCD, CDC, deduplication, reconciliation).
• Implement target-state designs aligned to lakehouse principles and validate during migrations.
• Support parallel-run phases and validate output consistency before cutover.
• Migrate workloads to modern data platforms preserving governance and control semantics.
• Re-implement ingestion, transformation, and orchestration pipelines on target platforms with audit and QA.
• Mentor engineers, review code/designs, and support decommissioning of legacy components.

Skills

Python

SQL

ETL/ELT

Data governance

Data modeling

Auditing

Tools

Azure Synapse Analytics

ADLS Gen2

Delta Lake

Azure Databricks

Unity Catalog

Terraform/ARM

CI/CD (Azure DevOps/GitHub Actions)

Docker

Job description

Allianz Technology seeks a Principal Data Engineer to hands-on re-engineer an enterprise data platform on Azure Synapse, with migration to Databricks. You will audit, reverse-engineer, and validate end-to-end data flows from ingestion to consumption, implementing lakehouse designs and ensuring governance and data quality.

The role involves parallel-run validation, complex data pipelines, and mentoring junior engineers.