Azure Data Engineer

Tata Consultancy Services; Thiruvananthapuram; On-site; INR 1,200,000 - 2,100,000; Full time

Job summary

Tata Consultancy Services in Trivandrum is looking for an Azure Data Engineer with 7 to 12 years of experience to design, build, and maintain scalable data solutions using Microsoft Fabric and Azure Databricks.

You will develop end-to-end ETL/ELT pipelines, implement Medallion architecture, and ensure data security and governance using Unity Catalog, RBAC, and enterprise standards. The role requires strong SQL, data modeling, and cost-aware Databricks management.

Qualifications

• Strong hands-on experience with Microsoft Fabric (Lakehouse, Data Factory, Dataflows Gen2, OneLake, Pipelines, Warehouse).
• Expertise in Azure Databricks and Spark architecture.
• Experience with Delta Lake, Medallion Architecture (Bronze, Silver, Gold).
• Strong SQL development skills including query optimization and complex transformations.
• Experience building end-to-end ETL/ELT pipelines using Azure Data Factory and Fabric Pipelines.
• Knowledge of ADLS Gen2, OneLake, and data ingestion patterns.
• Experience with Git, Azure DevOps, CI/CD deployments.
• Understanding of data modeling concepts (Star Schema, Snowflake Schema).

Responsibilities

• Design, develop, and maintain scalable data solutions using Microsoft Fabric and Azure Databricks.
• Build and optimize end-to-end ETL/ELT pipelines using Fabric Data Factory, Dataflows Gen2, and Azure Data Factory.
• Develop robust data transformation frameworks using PySpark, Spark SQL, and Delta Lake.
• Implement and manage Lakehouse architecture leveraging OneLake, Delta Tables, and Medallion Architecture (Bronze, Silver, Gold).
• Ingest batch and real-time data from multiple enterprise and third-party sources.
• Perform data modeling and design scalable data warehouses to support business analytics and reporting.
• Monitor, troubleshoot, and optimize Databricks jobs, clusters, notebooks, and Fabric workloads for performance and cost efficiency.
• Implement data quality controls, validation frameworks, and monitoring processes.
• Ensure data security, governance, and compliance using Unity Catalog, RBAC, and enterprise security standards.
• Build and manage CI/CD pipelines using Azure DevOps and Git for automated deployments.
• Provide production support, root cause analysis, and issue resolution for critical data platforms.
• Stay updated with emerging capabilities in Microsoft Fabric, Databricks, Spark, and Azure data services and drive adoption where applicable.

Skills

Microsoft Fabric

Azure Databricks

Spark architecture

Delta Lake

SQL development

Azure Data Factory

OneLake

Git/Azure DevOps

Data modeling (Star/Snowflake)

Education

BE/B.Tech/MCA/MSc/MS

Tools

Git

Azure DevOps

CI/CD pipelines

OneLake

Job description

Azure Data Engineer Opening

Greetings from TCS!

TCS has been a great pioneer in the technology arena and there's nothing that can stop us from growing together. Your role is of key importance, as it lays down the foundation for the entire project.

Make sure you have a valid EP number before interview.

To create an EP Number, please visit TCS iBegin

Kindly complete the registration if you have not done it yet.

Position: Azure Data Engineer

Experience: 7 to 12 years

Location: Trivandrum

Required Information

Details

Desired Competencies (Technical/Behavioral Competency)

Must-Have

1. Strong hands-on experience with Microsoft Fabric (Lakehouse, Data Factory, Dataflows Gen2, OneLake, Pipelines, Warehouse).
2. Expertise in Azure Databricks and Spark architecture.
3. Experience with Delta Lake, Medallion Architecture (Bronze, Silver, Gold).
4. Strong SQL development skills including query optimization and complex transformations.
5. Experience building end-to-end ETL/ELT pipelines using Azure Data Factory and Fabric Pipelines.
6. Knowledge of ADLS Gen2, OneLake, and data ingestion patterns.
7. Experience with Git, Azure DevOps, CI/CD deployments.
8. Understanding of data modeling concepts (Star Schema, Snowflake Schema).

Good-to-Have

1. Unity Catalog implementation and governance experience.
2. Experience with Direct Lake and Fabric Semantic Models.
3. Azure Synapse Analytics knowledge.
4. Real-time streaming using Spark Structured Streaming/Event Hub/Kafka.
5. Power BI development and optimization.
6. Exposure to AI/ML workloads on Databricks or Fabric.
7. Databricks Certified Data Engineer Associate/Professional certification.
8. Microsoft Fabric Data Engineer Associate (DP-700).

Role descriptions / Expectations from the Role

1. Design, develop, and maintain scalable data solutions using Microsoft Fabric and Azure Databricks.
2. Build and optimize end-to-end ETL/ELT pipelines using Fabric Data Factory, Dataflows Gen2, and Azure Data Factory.
3. Develop robust data transformation frameworks using PySpark, Spark SQL, and Delta Lake.
4. Implement and manage Lakehouse architecture leveraging OneLake, Delta Tables, and Medallion Architecture (Bronze, Silver, Gold).
5. Ingest batch and real-time data from multiple enterprise and third-party sources.
6. Perform data modeling and design scalable data warehouses to support business analytics and reporting.
7. Monitor, troubleshoot, and optimize Databricks jobs, clusters, notebooks, and Fabric workloads for performance and cost efficiency.
8. Implement data quality controls, validation frameworks, and monitoring processes.
9. Ensure data security, governance, and compliance using Unity Catalog, RBAC, and enterprise security standards.
10. Build and manage CI/CD pipelines using Azure DevOps and Git for automated deployments.
11. Provide production support, root cause analysis, and issue resolution for critical data platforms.
12. Stay updated with emerging capabilities in Microsoft Fabric, Databricks, Spark, and Azure data services and drive adoption where applicable.

Professional Skills: Strong analytical, problem-solving, communication, and ownership mindset. TCS Eligibility Criteria: *BE/B.tech/MCA/MSc./MS with minimum 7 years of relevant IT-experience post

Qualification. *Only Full-Time courses would be considered. *Candidates who have attended TCS interview in the last 1 months need not apply. Referrals are always welcome!!!Kindly don't apply if already attended interview in last 1 months.

Thanks and Regards,

Athira p s