Azure Data Engineering / Data Platform

Tata Consultancy Services; Chennai District; On-site; INR 1,300,000 - 1,900,000; Full time

Job summary

Tata Consultancy Services in Chennai is seeking a data engineering professional to design, develop, and maintain scalable data ingestion and transformation pipelines on Azure.

You will build ETL/ELT workflows using Azure Data Factory and PySpark, manage data solutions with Databricks and ADLS, and contribute to governance and DevOps practices.

Responsibilities

• Design, develop, and maintain scalable data ingestion and transformation pipelines on Azure.
• Build and optimize ETL/ELT workflows using Azure Data Factory and PySpark.
• Develop and manage data solutions using Azure Databricks and Azure Data Lake Storage (ADLS).
• Design and implement enterprise-scale data warehouse and analytics solutions.
• Develop event-driven and streaming data processing solutions using Azure services.
• Integrate applications and services using Azure Functions and Logic Apps.
• Create reusable, high-performance, and secure data engineering frameworks.
• Participate in data migration initiatives, including source-to-target mapping and reconciliation.
• Collaborate with architects, analysts, and cross-functional teams to deliver data solutions.
• Follow Agile methodologies and DevOps best practices for continuous delivery and deployment.
• Contribute to data governance, cataloging, and lineage initiatives.

Job description

Role & responsibilities

• Design, develop, and maintain scalable data ingestion and transformation pipelines on Azure.


• Build and optimize ETL/ELT workflows using Azure Data Factory and PySpark.


• Develop and manage data solutions using Azure Databricks and Azure Data Lake Storage (ADLS).


• Design and implement enterprise-scale data warehouse and analytics solutions.


• Develop event-driven and streaming data processing solutions using Azure services.


• Integrate applications and services using Azure Functions and Logic Apps.


• Create reusable, high-performance, and secure data engineering frameworks.


• Participate in data migration initiatives, including source-to-target mapping and reconciliation.


• Collaborate with architects, analysts, and cross-functional teams to deliver data solutions.


• Follow Agile methodologies and DevOps best practices for continuous delivery and deployment.


• Contribute to data governance, cataloging, and lineage initiatives.