Senior Data Engineer - Azure & Fabric Lakehouse Solutions

Datavail; United States; Remote; USD 140,000 - 180,000; Full time

Job summary

Datavail is seeking an experienced data engineer to design, build and maintain production-grade data pipelines across Microsoft Fabric and Azure. You will implement CDC, upserts, late-arriving data and god-tier reprocessing, while advancing dimensional models and Data Vault patterns.

You will lead complex delivery engagements with clients and collaborate across engineering teams. You will develop PySpark, Spark SQL and T-SQL transformations on Delta Lake layers and deploy Fabric solutions using

Qualifications

• 8+ years in data engineering/data warehousing.
• 3+ years delivering Azure solutions in production.
• Advanced T-SQL, Spark SQL, Python/PySpark; working knowledge of KQL and DAX.
• Delta Lake, Spark tuning, medallion/lakehouse patterns, CDC and incremental loading.
• Strong dimensional modeling and Data Vault 2.0.
• CI/CD, Git, automated testing, infrastructure as code, DataOps, governance and security.
• Professional-services/client-facing delivery experience.
• Authorization to work in the US and ability to travel up to 15%.
• Preferred: DP-700, DP-600, DP-203; streaming/event-driven, RAG/GenAI, Fabric IQ/data agents.

Responsibilities

• Design, build and maintain production-grade batch, incremental, real-time and near-real-time data pipelines across Microsoft Fabric and Azure.
• Implement CDC, watermarking, merge/upsert, late-arriving-data and idempotent reprocessing patterns.
• Develop PySpark, Spark SQL and T-SQL transformations across bronze, silver and gold Delta Lake layers.
• Build Fabric solutions using OneLake, Lakehouse, Warehouse, Data Factory, Dataflows Gen2, Spark notebooks, Real-Time Intelligence and Power BI.
• Implement dimensional and Data Vault models, data-quality testing, reconciliation, observability, logging, alerting and SLA monitoring.
• Establish Git-based CI/CD and automated deployment using Azure DevOps or GitHub, Fabric deployment pipelines, REST APIs, Bicep and Terraform.
• Apply Purview catalog/lineage, least-privilege access, sensitivity controls, encryption and secure connectivity.
• Support migrations from SQL Server, SSIS, SSAS, legacy warehouses and Synapse to Fabric/Azure, including conversion, reconciliation and cutover.
• Prepare governed data for AI/ML and RAG workloads, including ingestion, chunking, embeddings, indexing and telemetry.
• Lead engineering workstreams, estimates, risk management, client communication, mentoring, reusable accelerators and operational handover.

Skills

Azure Data Services

Spark SQL

Python / PySpark

T-SQL

KQL

DAX

Delta Lake

Data Vault 2.0

Dimensional modeling

Data governance

CI/CD

Git

Infrastructure as Code

DataOps

Security

Tools

Microsoft Fabric

Power BI

Azure DevOps / GitHub

OneLake

Data Factory

Job description

Datavail is seeking an experienced data engineer to design, build and maintain production-grade data pipelines across Microsoft Fabric and Azure. You will implement CDC, upserts, late-arriving data and god-tier reprocessing, while advancing dimensional models and Data Vault patterns.

You will lead complex delivery engagements with clients and collaborate across engineering teams. You will develop PySpark, Spark SQL and T-SQL transformations on Delta Lake layers and deploy Fabric solutions using