Assistant Manager - Data Engineering

LatentView Analytics Ltd.; Hyderabad; On-site; INR 3,000,000 - 5,400,000; Full time

Job summary

LatentView Analytics Ltd. in Hyderabad is seeking a Fabric Data Engineer with 7+ years of experience to drive the technical execution of enterprise data platform initiatives using Microsoft Fabric.

You will design and build scalable data pipelines and mentor a team of data engineers. The role focuses on implementing Lakehouse and Medallion Architecture, migrating data from Azure Synapse, SQL Server, Databricks, and Snowflake, and optimizing Fabric workloads while guiding governance and CI/CD

Qualifications

• 7+ years of experience in data engineering, data platform development, or ETL architecture.
• 1.5+ years hands-on experience with Microsoft Fabric.
• Strong expertise in Fabric components: OneLake, Lakehouse, Warehouse, Data Factory, and Real-Time Intelligence.

Responsibilities

• Lead design, development, and deployment of scalable data pipelines using Fabric Data Factory, Dataflows Gen2, and Spark Notebooks.
• Implement and optimize Medallion Architecture (Bronze/Silver/Gold) using Delta Lake, OneLake, and partitioning strategies.
• Drive data migration from Azure Synapse, SQL Server, Databricks, and Snowflake into Microsoft Fabric.
• Build batch, streaming, and real-time data pipelines using Fabric Real-Time Intelligence, Eventstreams, KQL, and Eventhouse.
• Design and optimize data models and Power BI semantic layers with Direct Lake, Import, and DirectQuery.
• Establish data governance, security, CI/CD, Git integration, automated testing, and deployment standards (Purview).
• Provide technical leadership and mentorship to data engineers via code reviews and best practices.

Skills

Microsoft Fabric

Python/PySpark

SQL/T-SQL

Spark

Delta Lake

OneLake

Lakehouse

Medallion Architecture

Data Factory

Power BI

Azure CLI

PowerShell

GitHub Actions

Kimball modelling

Data modeling

Azure DevOps

Databricks

Snowflake

Azure Synapse

Tools

Azure Synapse

Databricks

Snowflake

Azure DevOps

Job description

We are looking for a Fabric Data Engineer with 7+ years of experience to drive the technical execution of enterprise data platform initiatives using Microsoft Fabric. The role involves designing and building scalable data pipelines, implementing Lakehouse and Medallion Architecture, leading migration initiatives, optimizing Fabric workloads, and providing technical guidance to a team of data engineers.

• Lead the design, development, and deployment of scalable data pipelines using Microsoft Fabric Data Factory, Dataflows Gen2, and Spark Notebooks.


• Implement and optimize Medallion Architecture (Bronze/Silver/Gold) using Delta Lake, OneLake, schema evolution, and partitioning strategies.


• Drive data migration and integration from Azure Synapse, SQL Server, Databricks, and Snowflake into Microsoft Fabric.


• Build batch, streaming, and real-time data pipelines using Fabric Real-Time Intelligence, Eventstreams, KQL Database, and Eventhouse.


• Design and optimize data models and Power BI semantic layers, leveraging Direct Lake, Import, and DirectQuery for high-performance analytics.


• Establish data governance, security, CI/CD, Git integration, automated testing, and deployment standards, including Microsoft Purview for lineage and cataloging.


• Provide technical leadership and mentorship to data engineers through code reviews, architectural guidance, documentation, and engineering best practices.

Required Skills

• 7+ years of experience in Data Engineering, Data Platform Development, or ETL Architecture.


• 1.5+ years of hands-on experience with Microsoft Fabric.


• Strong hands-on expertise in Microsoft Fabric, OneLake, Lakehouse, Warehouse, Data Factory, and Real-Time Intelligence.


• Strong programming skills in Python/PySpark and SQL/T-SQL.


• Knowledge of M, PowerShell, or Azure CLI.


• Strong understanding of Apache Spark, Delta Lake, Medallion Architecture, data lakehouse patterns, partitioning, and performance optimization.


• Expertise in data modeling, including Kimball methodology, dimensional modeling, and star schemas.


• Hands-on experience with Azure DevOps/GitHub Actions, Fabric Deployment Pipelines, Git, CI/CD, and Microsoft Purview.


• Experience with Power BI semantic models, DAX, Direct Lake, Import, and DirectQuery.