Azure Data Engineer

mroads; Hyderabad, Bengaluru; On-site; INR 1,200,000 - 2,100,000; Full time

Job summary

mroads Hyderabad is seeking a data engineer to design and develop large-scale data pipelines using Azure Data Factory and Databricks. You will create efficient ETL processes to load data into Azure Blob Storage and other targets while optimizing SQL queries for performance.

You will collaborate with cross-functional teams, validate data quality, and communicate progress to stakeholders as part of continuous data strategy delivery.

Qualifications

• Strong understanding of SQL concepts, including query optimization and performance tuning.
• Experience with Databricks, Azure Data Factory, and ETL pipeline development.
• Knowledge of data processing technologies such as data warehousing, data lakes, and big data analytics.
• Excellent problem-solving skills and attention to detail.
• Ability to work collaboratively in a team environment and communicate effectively with stakeholders.
• Strong analytical and critical thinking skills, with the ability to make informed decisions.

Responsibilities

• Design and develop large-scale data pipelines using Azure Data Factory and Databricks.
• Create efficient ETL processes to extract, transform, and load data from various sources into Azure Blob Storage and other target systems.
• Collaborate with cross-functional teams to identify business requirements and design solutions that meet those needs.
• Develop and maintain complex SQL queries to optimize database performance and troubleshoot issues.
• Implement data quality checks and validation processes to ensure accuracy and integrity of the data.
• Work closely with stakeholders to understand their data needs and provide timely updates on project progress.

Skills

SQL queries

Databricks

Azure Data Factory

ETL pipeline

SQL optimization

Data analytics

Tools

Databricks

Azure Data Factory

Job description

Roles and Responsibility

• Design and develop large-scale data pipelines using Azure Data Factory and Databricks.


• Create efficient ETL processes to extract, transform, and load data from various sources into Azure Blob Storage and other target systems.


• Collaborate with cross-functional teams to identify business requirements and design solutions that meet those needs.


• Develop and maintain complex SQL queries to optimize database performance and troubleshoot issues.


• Implement data quality checks and validation processes to ensure accuracy and integrity of the data.


• Work closely with stakeholders to understand their data needs and provide timely updates on project progress.

Job Requirements

• Strong understanding of SQL concepts, including query optimization and performance tuning.


• Experience with Databricks, Azure Data Factory, and ETL pipeline development.


• Knowledge of data processing technologies such as data warehousing, data lakes, and big data analytics.


• Excellent problem-solving skills and attention to detail.


• Ability to work collaboratively in a team environment and communicate effectively with stakeholders.


• Strong analytical and critical thinking skills, with the ability to make informed decisions.