Senior Azure Databricks Data Engineer lead

Tata Consultancy Services; Pune District, Chennai District, Bengaluru; On-site; INR 2,500,000 - 4,500,000; Full time

Job summary

Tata Consultancy Services is seeking a Senior Azure Databricks Data Engineer Lead in India to design and implement data pipelines using Azure Data Lake, Databricks, and ADF. The role requires hands-on SQL and PySpark/Scala and experience with batch and real-time ingestion.

Ideal candidates will have 6+ years of data engineering experience, with familiarity in Kafka, NoSQL/SQL engines, and Informatica tools. Knowledge of GenAI, Maestro schedulers, and CI/CD processes is a plus.

Qualifications

• 6+ years of experience in data engineering with strong expertise in Azure Data Lake, Databricks, and ADF.
• Hands-on expertise in SQL and PySpark or Scala for building complex ETL/ELT solutions.
• Proven track record with batch & real-time ingestion, Kafka, and large-scale big data processing.
• Experience with relational and NoSQL engines: SQL Server, Oracle, Azure SQL, and Cosmos DB.
• Experience with Informatica PowerCenter, Informatica IDMC, or equivalent cloud data integration tools.
• Insurance domain knowledge (Policy, Claims, Billing, Underwriting, Actuarial, Regulatory data).
• Experience with Azure DevOps (AzDO) and GitHub-based CI/CD pipelines.
• Practical usage of Copilot for data engineering and development productivity.
• Exposure to Generative AI / GenAI use cases in data platforms and analytics.
• Familiarity with Maestro or similar enterprise schedulers, and data tools like Precisely or equivalent.

Responsibilities

• 6+ years of experience in data engineering with strong Azure Data Lake, Databricks, and ADF expertise.
• Build and optimize ETL/ELT pipelines using SQL and PySpark/Scala.
• Implement batch and real-time ingestion with Kafka and large-scale data processing.
• Work with relational and NoSQL engines such as SQL Server, Oracle, Azure SQL, Cosmos DB.
• Integrate data with Informatica PowerCenter/IDMC or equivalent cloud tools.
• Collaborate on CI/CD pipelines using Azure DevOps and GitHub.

Skills

Azure Data Lake

Databricks

ADF

SQL

PySpark/Scala

Kafka

Tools

Informatica PowerCenter

Informatica IDMC

Azure DevOps

GitHub Actions

Job description

Virtual drive on 21 Aug 2026 (friday)

Role :

Senior Azure Databricks Data Engineer Lead

Experience:

6- 10 yr

Location:

Chennai, Bangalore, Pune, Hyderabad

Responsibility:

• 6+ years of experience in data engineering with strong expertise in Azure Data Lake, Databricks, and ADF.
• Hands-on expertise in SQL and PySpark or Scala for building complex ETL/ELT solutions.
• Proven track record with batch & real-time ingestion, Kafka, and large-scale big data processing.
• Experience with relational and NoSQL engines: SQL Server, Oracle, Azure SQL, and Cosmos DB.
• Experience with Informatica PowerCenter, Informatica IDMC, or equivalent cloud data integration tools.
• Insurance domain knowledge (Policy, Claims, Billing, Underwriting, Actuarial, Regulatory data).
• Experience with Azure DevOps (AzDO) and GitHub-based CI/CD pipelines.
• Practical usage of Copilot for data engineering and development productivity.
• Exposure to Generative AI / GenAI use cases in data platforms and analytics.
• Familiarity with Maestro or similar enterprise schedulers, and data tools like Precisely or equivalent.