Data Engineer - AZURE

ESP Engineered; Hyderabad; On-site; INR 1,000,000 - 1,500,000; Full time

Job summary

ESP Engineered is seeking an experienced Azure Data Engineer in Hyderabad, India. The role involves designing and maintaining data pipelines using Azure technologies like Data Factory and Databricks. Ideal candidates should have over 5 years of hands-on experience with Azure services and a background in building ETL/ELT pipelines and data lakes. This position offers the opportunity to collaborate closely with data teams, ensuring scalable and efficient data solutions.

Qualifications

• 5+ years of hands-on experience as a Data Engineer with strong exposure to Azure services.
• Experience building ETL/ELT pipelines, data warehouses, and data lakes on Azure.
• Hands-on experience with Azure Data Factory, Databricks, ADLS Gen2, Azure SQL, and Synapse Analytics.

Responsibilities

• Design, develop, and maintain scalable data pipelines using Azure Data Factory, Databricks, SQL, and Python.
• Migrate on-premise and legacy data systems to Azure Data Lake and Azure Synapse Analytics.
• Optimize data processing workflows for performance, reliability, and cost efficiency.
• Implement data validation, quality checks, and monitoring for production pipelines.

Skills

Python

ETL/ELT pipelines

Azure Data Factory

Databricks (PySpark)

SQL

Apache Spark

Azure DevOps

CI/CD pipelines

Data validation

Problem-solving

Tools

Azure Synapse Analytics

Azure Data Lake

ARM Templates

Terraform

Power BI

Docker

Kubernetes

Job description

We are looking for a skilled Azure Data Engineer with 5+ years of hands‑on experience in building, maintaining, and optimizing data pipelines and analytics solutions on Microsoft Azure. The ideal candidate will work closely with senior engineers, data scientists, and business teams to deliver reliable and scalable data solutions. This role focuses on development, optimization, and support of cloud‑based data platforms while following best practices in data engineering and DevOps.

Technical Skills

• Python.

Key Responsibilities

• Design, develop, and maintain scalable data pipelines using Azure Data Factory, Databricks (PySpark), SQL, and Python.
• Assist in migrating on‑premise and legacy data systems to Azure Data Lake and Azure Synapse Analytics.
• Optimize data processing workflows for performance, reliability, and cost efficiency.
• Implement data validation, quality checks, and monitoring for production pipelines.
• Build and support batch and near real‑time data processing solutions using Event Hub and Stream Analytics.
• Collaborate with data analysts and data scientists to deliver curated and analytics‑ready datasets.
• Translate business requirements into technical data solutions under guidance from senior architects.
• Implement security, governance, and access control using Azure‑native tools and best practices.
• Support CI/CD pipelines and infrastructure automation using Azure DevOps, GitHub, ARM, or Terraform.
• Participate in code reviews and contribute to documentation and knowledge sharing.
• Troubleshoot and resolve data pipeline failures and performance issues.

Requirements

• 5+ years of hands‑on experience as a Data Engineer with strong exposure to Azure services.
• Experience building ETL/ELT pipelines, data warehouses, and data lakes on Azure.
• Hands‑on experience with Azure Data Factory, Databricks, ADLS Gen2, Azure SQL, and Synapse Analytics.
• Strong knowledge of Apache Spark, PySpark, SQL, and Python.
• Experience with modern data architectures such as Lakehouse and event‑driven systems.
• Exposure to Infrastructure as Code using ARM Templates or Terraform.
• Working knowledge of CI/CD pipelines using Azure DevOps or GitHub Actions.
• Familiarity with Azure Purview and role‑based access control (RBAC).
• Strong analytical and problem‑solving skills.
• Good communication skills and ability to work collaboratively in a team environment.

Preferred Skills

• Exposure to Power BI, Snowflake, or other analytical tools.
• Basic understanding of streaming technologies like Kafka or Delta Lake.
• Experience with Docker, Kubernetes, or AKS is a plus.
• Willingness to learn new Azure services and data engineering best practices.