Fabric Senior Data Engineer

EXL; Pune District; On-site; INR 1,200,000 - 1,800,000; Full time

Job summary

EXL is seeking an experienced Data Engineer to design and manage Azure/Microsoft Fabric Lakehouse solutions across Bronze, Silver, and Gold layers. You will build scalable data pipelines, implement CDC frameworks, and optimize Spark/PySpark workloads.

You will work with Power BI semantic models, Direct Lake, and data governance tools to ensure quality, security, and compliant data delivery for business stakeholders.

Qualifications

• 5+ years of data engineering experience with Azure/Microsoft Fabric.
• Hands-on with Fabric Lakehouse, OneLake, and Delta Lake.
• Proficient in Spark, PySpark, Python and SQL.
• Experience with CDC, ETL/ELT, and Medallion Architecture.
• Knowledge of Power BI semantic models and Direct Lake.
• Familiar with Purview, RBAC, CI/CD, and deployment.
• Commercial insurance data experience preferred.

Responsibilities

• Develop and maintain Microsoft Fabric / OneLake Lakehouse solutions across Bronze, Silver, and Gold layers.
• Build reusable data ingestion pipelines and CDC frameworks for sources including GCP, CSV, Excel, email files, and SharePoint.
• Implement Bronze-layer ingestion with audit logging, schema validation, schema-drift detection, and error handling.
• Develop Silver-layer transformations for cleansing, standardization, data quality, validation, and exception handling.
• Build Gold-layer datasets including curated entities, conformed dimensions, business metrics, KPIs, aggregations, and secure views.
• Develop and optimize Spark / PySpark, Python, and SQL workloads for scalable data processing.
• Build data pipelines using Fabric Data Pipelines, Synapse Pipelines, and Notebooks.
• Support Power BI semantic models and Direct Lake data consumption requirements.
• Implement data quality checks, reconciliation processes, monitoring, and operational controls.
• Follow data governance and security standards, including Purview, RBAC, PII/PHI controls, classification, and lineage.
• Troubleshoot pipeline failures, performance issues, data discrepancies, and production incidents.
• Collaborate with Solution Architects, Lead Data Engineers, BI teams, and business stakeholders to deliver scalable data solutions.

Skills

Azure Fabric

Spark

PySpark

Python

SQL

Delta Lake

CDC

ETL/ELT

Data Pipelines

CI/CD

Purview

RBAC

Power BI

Notebooks

Tools

Microsoft Fabric Lakehouse

OneLake

Notebooks

Microsoft Purview

Job description

Key Responsibilities

• Develop and maintain Microsoft Fabric / OneLake Lakehouse solutions across Bronze, Silver, and Gold layers.
• Build reusable data ingestion pipelines and CDC frameworks for sources including GCP, CSV, Excel, email files, and SharePoint.
• Implement Bronze-layer ingestion with audit logging, schema validation, schema-drift detection, and error handling.
• Develop Silver-layer transformations for cleansing, standardization, data quality, validation, and exception handling.
• Build Gold-layer datasets including curated entities, conformed dimensions, business metrics, KPIs, aggregations, and secure views.
• Develop and optimize Spark / PySpark, Python, and SQL workloads for scalable data processing.
• Build data pipelines using Fabric Data Pipelines, Synapse Pipelines, and Notebooks.
• Support Power BI semantic models and Direct Lake data consumption requirements.
• Implement data quality checks, reconciliation processes, monitoring, and operational controls.
• Follow data governance and security standards, including Purview, RBAC, PII/PHI controls, classification, and lineage.
• Troubleshoot pipeline failures, performance issues, data discrepancies, and production incidents.
• Collaborate with Solution Architects, Lead Data Engineers, BI teams, and business stakeholders to deliver scalable data solutions.

Required Experience & Skills

• 5+ years of overall Data Engineering experience with hands-on experience in Azure / Microsoft Fabric.
• Strong hands-on experience with Microsoft Fabric Lakehouse, OneLake, Warehouse, Data Pipelines, and Notebooks.
• Proficiency in Spark / PySpark, Python, SQL, and Delta Lake.
• Experience with ETL/ELT, CDC, data ingestion, transformation, and Medallion Architecture.
• Experience developing data quality, validation, reconciliation, and exception-handling frameworks.
• Knowledge of Power BI semantic models and Direct Lake.
• Experience with Microsoft Purview, Azure DevOps, Git, CI/CD, and deployment.
• Commercial insurance or brokerage data experience preferred.