Senior Azure Data Engineer

Tata Consultancy Services; Hyderabad; On-site; INR 2,500,000 - 4,000,000; Full time

Job summary

Tata Consultancy Services in Hyderabad is seeking a Senior Azure Databricks Data Engineer lead with 6-10 years of experience to design and implement large-scale data pipelines on Azure.

The role requires hands-on expertise in Azure Data Lake, Databricks, ADF, SQL, PySpark/Scala, Kafka, and experience with batch and real-time ingestion; you will lead complex ETL/ELT solutions and work with both relational and NoSQL data stores.

Qualifications

• 8+ years of experience in data engineering with Azure data stack.
• Hands-on expertise in SQL and PySpark/Scala for ETL/ELT solutions.
• Experience with batch and real-time ingestion, Kafka, and large-scale data processing.
• Experience with relational and NoSQL engines such as SQL Server, Oracle, Azure SQL, Cosmos DB.

Skills

SQL

PySpark

Scala

Kafka

Batch & real-time ingestion

Tools

Azure Databricks

Azure Data Lake

Azure Data Factory (ADF)

SQL Server

Oracle

Azure SQL

Cosmos DB

Informatica PowerCenter

Informatica IDMC

Job description

Role: Senior Azure Databricks Data Engineer lead

Experience: 6-10 Years

Requirements:

• 8+ years of experience in data engineering with strong expertise in Azure Data Lake, Databricks, and ADF.
• Hands-on expertise in SQL and PySpark or Scala for building complex ETL/ELT solutions.
• Proven track record with batch & real-time ingestion, Kafka, and large-scale big data processing.
• Experience with relational and NoSQL engines: SQL Server, Oracle, Azure SQL, and Cosmos DB.
• Experience with Informatica PowerCenter, Informatica IDMC, or equivalent cloud data integration tools.