Fabric Senior Data Engineer

EXL; Gurugram District; On-site; INR 1,000,000 - 2,000,000; Full time

Job summary

EXL is seeking a Senior Data Engineer to design, build, and optimize data pipelines and Lakehouse solutions on Microsoft Fabric across Bronze, Silver, and Gold layers. You will implement ingest, transformations, and analytics-ready datasets while ensuring data quality and security standards.

The role requires 5+ years of data engineering, strong PySpark/Python/SQL skills, and experience with Purview, CI/CD, and Power BI integrations. Based in India, suitable for an on-site work setup.

Qualifications

• 5+ years of overall Data Engineering experience with hands-on Azure / Microsoft Fabric.
• Strong hands-on experience with Microsoft Fabric Lakehouse, OneLake, Warehouse, Data Pipelines, and Notebooks.
• Proficiency in Spark / PySpark, Python, SQL, and Delta Lake.
• Experience with ETL/ELT, CDC, data ingestion, transformation, and Medallion Architecture.
• Experience developing data quality, validation, reconciliation, and exception-handling frameworks.
• Knowledge of Power BI semantic models and Direct Lake.
• Experience with Microsoft Purview, Azure DevOps, Git, CI/CD, and deployment.
• Commercial insurance or brokerage data experience preferred.

Responsibilities

• Develop and maintain Microsoft Fabric / OneLake Lakehouse solutions across Bronze, Silver, and Gold layers.
• Build reusable data ingestion pipelines and CDC frameworks for sources including GCP, CSV, Excel, email files, and SharePoint.
• Implement Bronze-layer ingestion with audit logging, schema validation, schema-drift detection, and error handling.
• Develop Silver-layer transformations for cleansing, standardization, data quality, validation, and exception handling.
• Build Gold-layer datasets including curated entities, conformed dimensions, business metrics, KPIs, aggregations, and secure views.
• Develop and optimize Spark / PySpark, Python, and SQL workloads for scalable data processing.
• Build data pipelines using Fabric Data Pipelines, Synapse Pipelines, and Notebooks.
• Support Power BI semantic models and Direct Lake data consumption requirements.
• Implement data quality checks, reconciliation processes, monitoring, and operational controls.
• Follow data governance and security standards, including Purview, RBAC, PII/PHI controls, classification, and lineage.
• Troubleshoot pipeline failures, performance issues, data discrepancies, and production incidents.
• Collaborate with Solution Architects, Lead Data Engineers, BI teams, and business stakeholders to deliver scalable data solutions.

Skills

5+ years Data Eng

Microsoft Fabric Lakehouse

OneLake

Spark / PySpark

Python

SQL

Delta Lake

ETL/ELT

CDC

Data ingestion

Medallion Architecture

Data quality

Validation

Reconciliation

Exception handling

Power BI semantic models

Direct Lake

Microsoft Purview

Azure DevOps

Git

CI/CD

Deployment

Insurance data experience

Tools

Notebooks

GCP

SharePoint

Job description

Job Description:

• Key Responsibilities


• Develop and maintain Microsoft Fabric / OneLake Lakehouse solutions across Bronze, Silver, and Gold layers.


• Build reusable data ingestion pipelines and CDC frameworks for sources including GCP, CSV, Excel, email files, and SharePoint.


• Implement Bronze-layer ingestion with audit logging, schema validation, schema-drift detection, and error handling.


• Develop Silver-layer transformations for cleansing, standardization, data quality, validation, and exception handling.


• Build Gold-layer datasets including curated entities, conformed dimensions, business metrics, KPIs, aggregations, and secure views.


• Develop and optimize Spark / PySpark, Python, and SQL workloads for scalable data processing.


• Build data pipelines using Fabric Data Pipelines, Synapse Pipelines, and Notebooks.


• Support Power BI semantic models and Direct Lake data consumption requirements.


• Implement data quality checks, reconciliation processes, monitoring, and operational controls.


• Follow data governance and security standards, including Purview, RBAC, PII/PHI controls, classification, and lineage.


• Troubleshoot pipeline failures, performance issues, data discrepancies, and production incidents.


• Collaborate with Solution Architects, Lead Data Engineers, BI teams, and business stakeholders to deliver scalable data solutions.

• Required Experience & Skills


• 5+ years of overall Data Engineering experience with hands‑on experience in Azure / Microsoft Fabric.


• Strong hands‑on experience with Microsoft Fabric Lakehouse, OneLake, Warehouse, Data Pipelines, and Notebooks.


• Proficiency in Spark / PySpark, Python, SQL, and Delta Lake.


• Experience with ETL/ELT, CDC, data ingestion, transformation, and Medallion Architecture.


• Experience developing data quality, validation, reconciliation, and exception‑handling frameworks.


• Knowledge of Power BI semantic models and Direct Lake.


• Experience with Microsoft Purview, Azure DevOps, Git, CI/CD, and deployment.


• Commercial insurance or brokerage data experience preferred.

Responsibilities

• Key Responsibilities


• Develop and maintain Microsoft Fabric / OneLake Lakehouse solutions across Bronze, Silver, and Gold layers.


• Build reusable data ingestion pipelines and CDC frameworks for sources including GCP, CSV, Excel, email files, and SharePoint.


• Implement Bronze-layer ingestion with audit logging, schema validation, schema-drift detection, and error handling.


• Develop Silver-layer transformations for cleansing, standardization, data quality, validation, and exception handling.


• Build Gold-layer datasets including curated entities, conformed dimensions, business metrics, KPIs, aggregations, and secure views.


• Develop and optimize Spark / PySpark, Python, and SQL workloads for scalable data processing.


• Build data pipelines using Fabric Data Pipelines, Synapse Pipelines, and Notebooks.


• Support Power BI semantic models and Direct Lake data consumption requirements.


• Implement data quality checks, reconciliation processes, monitoring, and operational controls.


• Follow data governance and security standards, including Purview, RBAC, PII/PHI controls, classification, and lineage.


• Troubleshoot pipeline failures, performance issues, data discrepancies, and production incidents.


• Collaborate with Solution Architects, Lead Data Engineers, BI teams, and business stakeholders to deliver scalable data solutions.

• Required Experience & Skills


• 5+ years of overall Data Engineering experience with hands‑on experience in Azure / Microsoft Fabric.


• Strong hands‑on experience with Microsoft Fabric Lakehouse, OneLake, Warehouse, Data Pipelines, and Notebooks.


• Proficiency in Spark / PySpark, Python, SQL, and Delta Lake.


• Experience with ETL/ELT, CDC, data ingestion, transformation, and Medallion Architecture.


• Experience developing data quality, validation, reconciliation, and exception‑handling frameworks.


• Knowledge of Power BI semantic models and Direct Lake.


• Experience with Microsoft Purview, Azure DevOps, Git, CI/CD, and deployment.


• Commercial insurance or brokerage data experience preferred.

Qualifications

• Key Responsibilities


• Develop and maintain Microsoft Fabric / OneLake Lakehouse solutions across Bronze, Silver, and Gold layers.


• Build reusable data ingestion pipelines and CDC frameworks for sources including GCP, CSV, Excel, email files, and SharePoint.


• Implement Bronze-layer ingestion with audit logging, schema validation, schema-drift detection, and error handling.


• Develop Silver-layer transformations for cleansing, standardization, data quality, validation, and exception handling.


• Build Gold-layer datasets including curated entities, conformed dimensions, business metrics, KPIs, aggregations, and secure views.


• Develop and optimize Spark / PySpark, Python, and SQL workloads for scalable data processing.


• Build data pipelines using Fabric Data Pipelines, Synapse Pipelines, and Notebooks.


• Support Power BI semantic models and Direct Lake data consumption requirements.


• Implement data quality checks, reconciliation processes, monitoring, and operational controls.


• Follow data governance and security standards, including Purview, RBAC, PII/PHI controls, classification, and lineage.


• Troubleshoot pipeline failures, performance issues, data discrepancies, and production incidents.


• Collaborate with Solution Architects, Lead Data Engineers, BI teams, and business stakeholders to deliver scalable data solutions.

• Required Experience & Skills


• 5+ years of overall Data Engineering experience with hands‑on experience in Azure / Microsoft Fabric.


• Strong hands‑on experience with Microsoft Fabric Lakehouse, OneLake, Warehouse, Data Pipelines, and Notebooks.


• Proficiency in Spark / PySpark, Python, SQL, and Delta Lake.


• Experience with ETL/ELT, CDC, data ingestion, transformation, and Medallion Architecture.


• Experience developing data quality, validation, reconciliation, and exception‑handling frameworks.


• Knowledge of Power BI semantic models and Direct Lake.


• Experience with Microsoft Purview, Azure DevOps, Git, CI/CD, and deployment.


• Commercial insurance or brokerage data experience preferred.

Requirements: