Azure Databricks & Integration Engineer

Tata Consultancy Services; Hyderabad, Chennai District, Bengaluru; On-site; INR 1,400,000 - 2,100,000; Full time

Job summary

Tata Consultancy Services is seeking an Azure Databricks & Integration Engineer with 6-10 years of data engineering experience to design, develop, and support enterprise-scale data solutions on Azure.

The role focuses on Azure Databricks, Azure Data Factory, ETL/ELT processes, PySpark, Spark SQL, Python, and cloud-native architectures, collaborating with data teams and stakeholders to deliver scalable data pipelines with quality, security, and governance.

Qualifications

• 6-10 years of experience in Data Engineering and Integration.
• Hands-on Azure Databricks expertise.
• Proficiency with Azure Data Factory (ADF) and data integration.
• Strong knowledge of PySpark, Spark SQL, Python and SQL.
• Experience with ETL/ELT development and data transformation.
• Knowledge of Azure Data Lake, Azure Storage and cloud-native architectures.
• Experience with DevOps, CI/CD pipelines, and Git-based version control.
• Strong problem-solving and analytical skills.

Responsibilities

• Design scalable data pipelines using Azure Databricks and Azure integration services.
• Build and optimize ETL/ELT solutions for ingesting, transforming, and processing large datasets.
• Develop data integration workflows using Azure Data Factory (ADF) and related Azure services.
• Implement data processing solutions using PySpark, Spark SQL, and Databricks.
• Work closely with architects, business stakeholders, and data teams to deliver data-driven solutions.
• Ensure data quality, performance, security, and governance standards.
• Troubleshoot and optimize existing data platforms and integration processes.
• Support CI/CD implementation and cloud deployment best practices.

Skills

Azure Databricks

Azure Data Factory

PySpark

Spark SQL

Python

SQL

ETL/ELT

Cloud data architecture

CI/CD

Git

Tools

Azure Data Factory

Delta Lake

Databricks Lakehouse

Job description

Azure Databricks & Integration Engineer

Experience: 6-10 YearsLocations: Chennai, Bengaluru, Hyderabad, Pune, Kochi, Kolkata, NCR/Delhi

Interview Mode: Virtual InterviewInterview Date: 8th August 2026 (Saturday)

Job Description

We are looking for a skilled Azure Databricks & Integration Engineer with experience in designing, developing, and supporting enterprise-scale data integration and analytics solutions on the Azure platform. The ideal candidate should have expertise in Azure Databricks, Azure Data Factory, ETL/ELT processes, and cloud-based data engineering solutions.

Key Responsibilities

• Design and develop scalable data pipelines using Azure Databricks and Azure integration services.
• Build and optimize ETL/ELT solutions for ingesting, transforming, and processing large datasets.
• Develop data integration workflows using Azure Data Factory (ADF) and related Azure services.
• Implement data processing solutions using PySpark, Spark SQL, and Databricks.
• Work closely with architects, business stakeholders, and data teams to deliver data-driven solutions.
• Ensure data quality, performance, security, and governance standards.
• Troubleshoot and optimize existing data platforms and integration processes.
• Support CI/CD implementation and cloud deployment best practices.

Required Skills

• 6-10 years of experience in Data Engineering and Integration.
• Strong hands-on experience with Azure Databricks.
• Expertise in Azure Data Factory (ADF) and data integration solutions.
• Strong knowledge of PySpark, Spark SQL, Python, and SQL.
• Experience with ETL/ELT development and data transformation processes.
• Knowledge of Azure Data Lake, Azure Storage, and cloud-native architectures.
• Experience with DevOps, CI/CD pipelines, and Git-based version control.
• Strong problem-solving and analytical skills.

Preferred Skills

• Experience with Delta Lake and Databricks Lakehouse architecture.
• Exposure to Azure Synapse Analytics.
• Knowledge of streaming technologies such as Kafka and Event Hubs.
• Experience working in large-scale enterprise data environments.