Fabric Data Engineer

LatentView; Hyderabad; Hybrid; INR 3,500,000 - 7,500,000; Full time

Job summary

LatentView in India seeks a senior data engineer to lead design, development and deployment of scalable data pipelines using Microsoft Fabric Data Factory, Dataflows Gen2 and Spark Notebooks. You will implement Medallion Architecture with Delta Lake, OneLake, and partitioning strategies, and drive data migration from Azure Synapse, SQL Server, Databricks and Snowflake into Fabric.

Also you will build batch, streaming and real-time pipelines, design data models and Power BI semantics, and

Qualifications

• 7+ years of experience in Data Engineering, Data Platform Development, or ETL Architecture.
• 1.5+ years hands-on experience with Microsoft Fabric.
• Strong hands-on expertise in Microsoft Fabric, OneLake, Lakehouse, Warehouse, Data Factory, and Real-Time Intelligence.
• Strong programming skills in Python/PySpark and SQL/T-SQL.
• Knowledge of M, PowerShell, or Azure CLI.
• Strong understanding of Apache Spark, Delta Lake, Medallion Architecture, data lakehouse patterns, partitioning, and performance optimization.
• Expertise in data modeling, including Kimball methodology, dimensional modeling, star schemas.
• Hands-on experience with Azure DevOps/GitHub Actions, Fabric Deployment Pipelines, Git, CI/CD, and Microsoft Purview.
• Experience with Power BI semantic models, DAX, Direct Lake, Import, and DirectQuery

Responsibilities

• Lead design, development, and deployment of scalable data pipelines using Fabric Data Factory, Dataflows Gen2, and Spark Notebooks.
• Implement and optimize Medallion Architecture (Bronze/Silver/Gold) using Delta Lake, OneLake, schema evolution, and partitioning strategies.
• Drive data migration and integration from Azure Synapse, SQL Server, Databricks, and Snowflake into Fabric.
• Build batch, streaming, and real-time data pipelines using Fabric Real-Time Intelligence, Eventstreams, KQL Database, and Eventhouse.
• Design and optimize data models and Power BI semantic layers, leveraging Direct Lake, Import, and DirectQuery for high-performance analytics.
• Establish data governance, security, CI/CD, Git integration, automated testing, and deployment standards including Microsoft Purview.
• Provide technical leadership and mentorship to data engineers through code reviews, architectural guidance, documentation, and engineering best practices.

Skills

Python/PySpark

SQL/T-SQL

Apache Spark

Delta Lake

Medallion Architecture

Data Modeling

Kimball dimensional modeling

CI/CD

Git

Power BI

Fabric/OneLake

Real-Time Intelligence

Partitioning

Tools

Microsoft Fabric

OneLake

Lakehouse

Data Factory

Databricks

Azure DevOps

Purview

Job description

Role & responsibilities

• Lead the design, development, and deployment of scalable data pipelines using Microsoft Fabric Data Factory, Dataflows Gen2, and Spark Notebooks.
• Implement and optimize Medallion Architecture (Bronze/Silver/Gold) using Delta Lake, OneLake, schema evolution, and partitioning strategies.
• Drive data migration and integration from Azure Synapse, SQL Server, Databricks, and Snowflake into Microsoft Fabric.
• Build batch, streaming, and real-time data pipelines using Fabric Real-Time Intelligence, Eventstreams, KQL Database, and Eventhouse.
• Design and optimize data models and Power BI semantic layers, leveraging Direct Lake, Import, and DirectQuery for high-performance analytics.
• Establish data governance, security, CI/CD, Git integration, automated testing, and deployment standards, including Microsoft Purview for lineage and cataloging.
• Provide technical leadership and mentorship to data engineers through code reviews, architectural guidance, documentation, and engineering best practices.

Preferred candidate profile

• 7+ years of experience in Data Engineering, Data Platform Development, or ETL Architecture.
• 1.5+ years of hands-on experience with Microsoft Fabric.
• Strong hands-on expertise in Microsoft Fabric, OneLake, Lakehouse, Warehouse, Data Factory, and Real-Time Intelligence.
• Strong programming skills in Python/PySpark and SQL/T-SQL.
• Knowledge of M, PowerShell, or Azure CLI.
• Strong understanding of Apache Spark, Delta Lake, Medallion Architecture, data lakehouse patterns, partitioning, and performance optimization.
• Expertise in data modeling, including Kimball methodology, dimensional modeling, star schemas.
• Hands-on experience with Azure DevOps/GitHub Actions, Fabric Deployment Pipelines, Git, CI/CD, and Microsoft Purview.
• Experience with Power BI semantic models, DAX, Direct Lake, Import, and DirectQuery.