Azure / MS Fabric Data Engineer

Tata Consultancy Services Limited; Ernakulam, Kolkata District, Pune District; On-site; INR 1,200,000 - 1,800,000; Full time

Job summary

Tata Consultancy Services Limited in India seeks a data engineer to design and develop data pipelines using Medallion Architecture, PySpark, and Azure data services. You will convert SQL procedures to PySpark, optimize data loads from on-premises DWH, and collaborate with architects and product teams.

You will work with Azure Synapse, ADF, SQL functions, and data warehouse concepts, delivering scalable data solutions while maintaining strong stakeholder communication and agile practices.

Qualifications

• Experience with PySpark and notebooks.
• Hands-on with Azure Data Factory (ADF).
• Proficiency in Azure Synapse Analytics.
• Strong SQL & Stored Procedures knowledge.
• Experience converting SQL stored procedures to PySpark.
• Familiarity with Medallion Architecture (bronze/silver/gold).
• Knowledge of SQL aggregation functions.
• Understanding of Data Warehouse concepts.

Responsibilities

• Design and develop data solutions using Medallion Architecture.
• Load and process data from on-premises DWH and other sources into Fabric.
• Develop PySpark-based data processing solutions.
• Work with data aggregation and DWH techniques.
• Collaborate with technology, architecture, product, and design teams.
• Maintain strong communication and stakeholder collaboration skills.

Skills

PySpark & Notebooks

Azure Data Factory

Azure Synapse Analytics

SQL & Stored Procedures

SQL Stored Procedure to PySpark

Medallion Architecture

SQL Aggregation Functions

Data Warehouse Concepts

Tools

Microsoft Fabric / DP-600

Azure Data Engineering / DP-203

Power BI & Power Platform

Agile / JIRA

Job description

Must-Have Skills

• PySpark & Notebooks


• Azure Data Factory (ADF)


• Azure Synapse Analytics


• SQL & Stored Procedures


• SQL Stored Procedure to PySpark conversion


• Medallion Architecture


• SQL Aggregation Functions


• Data Warehouse (DWH) concepts

Good-to-Have

• Microsoft Fabric / DP-600


• Azure Data Engineering / DP-203


• Power BI & Power Platform


• Agile / JIRA

Responsibilities

• Design and develop data solutions using Medallion Architecture.


• Load and process data from on-premises DWH and other sources into Fabric.


• Develop PySpark-based data processing solutions.


• Work with data aggregation and DWH techniques.


• Collaborate with technology, architecture, product, and design teams.


• Maintain strong communication and stakeholder collaboration skills.