Azure Data Engineer

Mindpool Technologies; Hyderabad, Chennai District, Bengaluru; On-site; INR 2,800,000 - 4,400,000; Full time

Job summary

Mindpool Technologies is seeking an experienced Azure Data Engineer to design, develop, and maintain scalable data solutions on Microsoft Azure. The role emphasizes PySpark, ETL/ELT pipelines, and data migration testing to ensure data quality across large-scale migrations.

The ideal candidate will have 8-10 years of experience in data engineering, strong SQL skills, and hands-on expertise with Azure Data Factory, Synapse, Databricks, and PySpark.

Qualifications

• 8–10 years of experience in Azure data engineering and designing scalable data solutions.
• Hands-on expertise with Azure data services and big data processing.
• Strong skills in data migration testing, validation, and quality checks.

Responsibilities

• Design, develop, and optimize data pipelines using Microsoft Azure services.
• Develop scalable data processing solutions using PySpark.
• Perform data migration testing, including source-to-target validation, data reconciliation, and data quality checks.
• Collaborate with stakeholders to understand data requirements and governance.
• Build and maintain ETL/ELT workflows for structured and unstructured data.
• Monitor and troubleshoot data pipelines to ensure high availability and performance.
• Implement data quality, governance, and cloud security best practices.
• Participate in code reviews, testing, and deployment activities.

Skills

Data engineering

ETL/ELT

Analytical skills

Communication

Git & CI/CD

Tools

Azure Data Factory

Azure Data Lake Storage

Azure Synapse Analytics

Azure Databricks

PySpark

SQL

Job description

Role: Azure Data Engineer
Experience Required: 8-10
Location: ~BANGALORE~HYDERABAD~CHENNAI~
Skills: Digital: Microsoft Azure~Digital : Python~Azure Data Factory

Job Summary

We are looking for an experienced Azure Data Engineer with 810 years of experience in designing, developing, and maintaining scalable data solutions on Microsoft Azure. The ideal candidate should have strong expertise in Azure data services, PySpark, and data migration testing, with experience in building robust ETL/ELT pipelines and validating large-scale data migrations.

Key Responsibilities

• Design, develop, and optimize data pipelines using Microsoft Azure services.
• Develop scalable data processing solutions using PySpark.
• Perform data migration testing, including source-to-target validation, data reconciliation, and data quality checks.
• Collaborate with business stakeholders, data architects, and application teams to understand data requirements.
• Build and maintain ETL/ELT workflows for structured and unstructured data.
• Monitor and troubleshoot data pipelines to ensure high availability and performance.
• Implement data quality, governance, and security best practices.
• Participate in code reviews, testing, and deployment activities.

Required Skills

• Strong experience with Microsoft Azure data services (Azure Data Factory, Azure Data Lake Storage, Azure Synapse Analytics, Azure Databricks).
• Hands-on experience with PySpark for large-scale data processing.
• Expertise in data migration testing, data validation, reconciliation, and quality assurance.
• Strong SQL skills and experience with relational databases.
• Knowledge of ETL/ELT concepts and data warehousing.
• Familiarity with version control tools such as Git and CI/CD practices.
• Good analytical, problem-solving, and communication skills.

Preferred Skills

• Experience working in Agile/Scrum environments.
• Knowledge of Python and Azure DevOps.
• Understanding of data governance and cloud security best practices.
• Azure certifications are an added advantage.