Azure Data Engineer

Capgemini; Chennai District, Hyderabad; Hybrid; INR 800,000 - 1,200,000; Full time

Job summary

Capgemini is seeking an experienced Azure Data Engineer to join our team in Chennai, India. In this role, you will design, implement, and optimize modern data solutions utilizing Microsoft Azure cloud services for leading global brands.

Key responsibilities include developing end-to-end data pipelines, working with Azure SQL Database, and implementing CI/CD pipelines. The ideal candidate will have strong skills in Azure Data Factory, Azure Databricks, and data processing with Python and Spark.

Qualifications

• Experience with Azure Data Factory (ADF) and Azure Databricks (ADB) required.
• Proficiency in Python, PySpark, Scala, and Hive is necessary.
• Familiarity with cloud data architecture and ETL/ELT frameworks is beneficial.

Responsibilities

• Design and develop end-to-end data pipelines using Azure Data Factory and Azure Databricks.
• Work with Azure SQL Database, Azure SQL Data Warehouse, and Azure Synapse Analytics.
• Build and maintain distributed data workflows in Azure Databricks.

Skills

Azure Data Factory (ADF)

Azure Databricks (ADB)

Azure SQL Database

Python

PySpark

Scala

Hive

CI/CD pipeline development

Job description

About the Role

We are looking for an experienced and passionate Azure Data Engineer to join our team at Capgemini. You will work with leading global brands to design, implement, and optimize modern data solutions using Microsoft Azure cloud services.

Key Responsibilities

• Design and develop end-to-end data pipelines using Azure Data Factory (ADF) and Azure Databricks (ADB).
• Work with Azure SQL Database, Azure SQL Data Warehouse, and Azure Synapse Analytics.
• Develop data processing solutions using Python, PySpark, Scala, and Hive.
• Build and maintain distributed data workflows on Azure Databricks.
• Implement CI/CD pipelines for data applications.
• Collaborate with teams to gather requirements and deliver high-quality data solutions.
• Ensure data quality, performance tuning, and cloud data architecture best practices.

Required Skills & Experience

Primary Skills (At least one required)

• Azure Data Factory (ADF)
• Azure Databricks (ADB)

Technical Skills

• Azure Data Bricks, Azure Data Factory
• Azure SQL Database, Azure SQL Warehouse, Azure Synapse Analytics Python, PySpark, Scala, Hive
• CI/CD pipeline development in data environments
• Knowledge of cloud data architecture and ETL/ELT frameworks

Good to Have

• Experience with Azure DevOps
• Performance optimization in distributed processing
• Knowledge of data governance and security on Azure