Azure Data Engineer with Microsoft Fabric

Tata Consultancy Services; Mumbai; On-site; INR 1,800,000 - 3,200,000; Full time

Job summary

Tata Consultancy Services is hiring a data engineer to design and deliver end-to-end data pipelines using Microsoft Fabric and Azure data services in Mumbai. You will build data ingestion pipelines, implement lakehouse architectures, and create semantic models for BI consumption.

The role requires strong PySpark/SQL skills, experience with Fabric Dataflows Gen2, Lakehouse, Notebooks, and a solid grounding in data governance and security practices. Collaboration with multiple teams is essential.

Qualifications

• Experience in Data Engineering.
• Strong hands-on experience with Microsoft Fabric (mandatory).
• Proficiency in Azure Data Services: ADLS Gen2, ADF/Synapse, Azure SQL.
• Expertise in SQL, PySpark, Spark SQL, and Delta Lake.
• Experience with Pipelines, Dataflows Gen2, Lakehouse, Notebooks in Fabric.
• Understanding of data modeling, data warehousing, and semantic layer design.
• Working knowledge of DevOps practices, Git, CI/CD pipelines.
• Experience in data governance, Purview, lineage, and metadata management.
• Strong understanding of security: RBAC, Key Vault, encryption, PII handling.

Responsibilities

• Design and develop end-to-end data engineering solutions using Microsoft Fabric (Dataflows Gen2, Pipelines, Lakehouse, OneLake).
• Build and manage data ingestion pipelines using Azure services such as ADLS Gen2, Azure Data Factory, Azure Synapse, Event Hubs.
• Develop Notebooks using PySpark/Spark SQL for data transformation and validation.
• Implement and optimize Lakehouse architectures using Delta/Parquet formats.
• Create and maintain Semantic Models in Fabric for reporting and BI consumption.
• Implement secure access using RBAC, Key Vault, Managed Identity, and follow best practices for PII protection.
• Improve performance through partitioning, indexing, and workload tuning within Fabric.
• Build CI/CD pipelines using Git & Azure DevOps for data engineering workflows.
• Collaborate with business, analytics, and architecture teams to deliver scalable data solutions.

Skills

Data Engineering

Microsoft Fabric

Azure Data Services

SQL

PySpark

Spark SQL

Delta Lake

Data Modeling

DevOps

Data Governance

Security RBAC

Tools

Microsoft Fabric

Azure Data Factory

Azure Synapse

Job description

Key Responsibilities:

• Design and develop end-to-end data engineering solutions using Microsoft Fabric (Dataflows Gen2, Pipelines, Lakehouse, OneLake).
• Build and manage data ingestion pipelines using Azure services such as ADLS Gen2, Azure Data Factory, Azure Synapse, Event Hubs.
• Develop Notebooks using PySpark/Spark SQL for data transformation and validation.
• Implement and optimize Lakehouse architectures using Delta/Parquet formats.
• Create and maintain Semantic Models in Fabric for reporting and BI consumption.
• Implement secure access using RBAC, Key Vault, Managed Identity, and follow best practices for PII protection.
• Improve performance through partitioning, indexing, and workload tuning within Fabric.
• Build CI/CD pipelines using Git & Azure DevOps for data engineering workflows.
• Collaborate with business, analytics, and architecture teams to deliver scalable data solutions.

Required Skills & Experience:

• Experience in Data Engineering.
• Strong hands-on experience with Microsoft Fabric (mandatory).
• Proficiency in Azure Data Services: ADLS Gen2, ADF/Synapse, Azure SQL.
• Expertise in SQL, PySpark, Spark SQL, and Delta Lake.
• Experience with Pipelines, Dataflows Gen2, Lakehouse, Notebooks in Fabric.
• Understanding of data modeling, data warehousing, and semantic layer design.
• Working knowledge of DevOps practices, Git, CI/CD pipelines.
• Experience in data governance, Purview, lineage, and metadata management.
• Strong understanding of security: RBAC, Key Vault, encryption, PII handling.

Good to Have Skills:

• Certifications: DP203 (Azure Data Engineer) or DP600 (Fabric Analytics Engineer).
• Experience with Power BI (DAX, RLS, semantic model optimization).
• BFSI/Financial domain experience.
• Experience with Databricks or Synapse Spark environments.