Azure Data Engineer

Tata Consultancy Services; Delhi; On-site; INR 1,500,000 - 2,100,000; Full time

Job summary

Tata Consultancy Services in Delhi is seeking an experienced Azure Data Engineer to design, develop, and optimize scalable cloud data solutions using Azure Databricks and PySpark.

You will develop and maintain ETL/ELT workflows, work with ADF, ADLS Gen2, and Azure SQL, and ensure data quality, governance, and security across data platforms. Collaboration with stakeholders and teams is essential for success.

Qualifications

• Hands-on Azure Databricks and PySpark experience.
• Experience with ADF and ADLS Gen2.
• Proficiency in SQL and data modeling.
• Strong ETL, data warehousing, and big data processing knowledge.
• Experience with Git, CI/CD, and Agile methodologies.

Responsibilities

• Design and build scalable data pipelines using Azure Databricks and PySpark.
• Develop and maintain ETL/ELT workflows for batch and real-time data processing.
• Work with ADF, ADLS Gen2, and Azure SQL services.
• Optimize data processing performance and ensure data quality and reliability.
• Collaborate with business stakeholders, data analysts, and architects to deliver data solutions.
• Implement data governance, security, and best practices in cloud data platforms.

Skills

Azure Databricks

PySpark

SQL

ETL

Data warehousing

Big data processing

Git

CI/CD

Agile methodologies

Tools

Azure Data Factory (ADF)

ADLS Gen2

Azure SQL

Job description

We are looking for an experienced Azure Data Engineer with strong expertise in Azure Databricks, PySpark, and Azure Data Platform services to design, develop, and optimize scalable data solutions.

Key Responsibilities :

• Design and build scalable data pipelines using Azure Databricks and PySpark.
• Develop and maintain ETL/ELT workflows for batch and real-time data processing.
• Work with Azure Data Factory (ADF), Azure Data Lake Storage (ADLS Gen2), and Azure SQL services.
• Optimize data processing performance and ensure data quality and reliability.
• Collaborate with business stakeholders, data analysts, and architects to deliver data solutions.
• Implement data governance, security, and best practices in cloud data platforms.

Required Skills :

• Strong hands-on experience with Azure Databricks and PySpark.
• Experience with Azure Data Factory (ADF) and ADLS Gen2.
• Proficiency in SQL and data modeling concepts.
• Good understanding of ETL, data warehousing, and big data processing.
• Experience with Git, CI/CD, and Agile methodologies.

Preferred Skills :

• Experience with Delta Lake, Spark Optimization, and Data Lake architecture.
• Knowledge of Azure Synapse Analytics.
• Azure Certification would be an added advantage.