Senior Data Engineer - Azure Fabric & Delta Lake Expert

Datavail; United States; On-site; USD 150,000 - 210,000; Full time

Job summary

Datavail is seeking a senior data engineer to design and optimize end-to-end data pipelines on Microsoft Fabric and Azure. You will implement CDC, upserts, and real-time processing while ensuring data quality and observability.

Ideal candidates have 8+ years in data warehousing, Delta Lake mastery, and strong CI/CD experience with GitHub or Azure DevOps. Travel up to 15% is required for client delivery and engagements.

Qualifications

• 8+ years in data engineering and data warehousing with production Azure experience.
• Strong Delta Lake, medallion/lakehouse patterns, and CDC loading knowledge.
• Professional-services or client-facing delivery experience required.
• Authorization to work in the US and willingness to travel up to 15%.

Responsibilities

• Design, build and maintain production-grade data pipelines across Microsoft Fabric and Azure.
• Implement CDC, watermarking, merge/upsert, late-arriving-data and idempotent reprocessing patterns.
• Develop PySpark, Spark SQL and T-SQL transformations across Delta Lake layers.
• Lead engineering workstreams and client communication, mentoring, and reusable accelerators.

Skills

Data engineering

Azure solutions

Data pipelines

CI/CD

Data governance

Security

Client-facing delivery

Tools

Git & GitHub

Azure DevOps

Terraform

Bicep

Purview

Power BI

Data Factory

Spark notebooks

Delta Lake

OneLake

Lakehouse

DAX

KQL

Job description

Datavail is seeking a senior data engineer to design and optimize end-to-end data pipelines on Microsoft Fabric and Azure. You will implement CDC, upserts, and real-time processing while ensuring data quality and observability.

Ideal candidates have 8+ years in data warehousing, Delta Lake mastery, and strong CI/CD experience with GitHub or Azure DevOps. Travel up to 15% is required for client delivery and engagements.