Azure Data Engineer - Team Lead

ESP Engineered; Hyderabad; On-site; INR 1,200,000 - 2,000,000; Full time

Job summary

ESP Engineered is seeking an experienced Data Engineer in Hyderabad. The ideal candidate will have over 8 years' experience in designing, developing, and optimizing data solutions. Responsibilities include leading data engineering strategies, developing data pipelines, and implementing modern data architectures across cloud environments. Candidates should possess strong expertise in Python, SQL, Spark, and cloud platforms like AWS or Azure. A Bachelor's or Master's degree in a related field is required. A collaborative, problem-solving mindset is essential.

Qualifications

• 8+ years in Data Engineering or Data Architecture roles.
• Proven experience with cloud platforms (AWS, Azure, or GCP).
• Hands-on experience with Snowflake, Databricks, or Delta Lake.

Responsibilities

• Lead data engineering strategy and architect cloud-based data platforms.
• Develop and optimize data pipelines and ETL/ELT processes.
• Implement DevOps/Data Ops practices for automated testing and deployment.

Skills

Python

SQL

Spark

Kafka

Airflow

C#

.NET Core

Azure PaaS

REST API Development

Agile Practices

Education

Bachelor's or master's in computer science, Engineering, or related field

Tools

Snowflake

Databricks

Delta Lake

AWS

Azure

GCP

Job description

We are seeking an experienced Data Engineer with over 8 years of experience in designing, developing, and optimizing enterprise-scale data solutions. The ideal candidate will lead our data engineering strategy, architect cloud-based data platforms, and mentor engineering teams to deliver scalable and reliable data systems that power business intelligence, AI, and analytics.

Design and implement modern data architectures across cloud and hybrid environments.

• Develop and optimize data pipelines and ETL/ELT processes for high-volume, high-velocity data.
• Lead data platform modernization initiatives using cutting-edge tools and frameworks.
• Define and enforce data governance, security, and compliance standards.
• Partner with Data Science, BI, and Product teams to ensure data availability, accuracy, and performance.
• Provide technical leadership and mentorship to data engineers and developers.
• Implement DevOps/Data Ops practices for automated testing, deployment, and monitoring.
• Continuously evaluate and introduce innovative data technologies that enhance efficiency and scalability.
• Translate business requirements into robust and scalable technical data solutions.
• Support real-time and streaming data architectures for analytics and ML workloads.
• Designing and developing REST APIs / Azure App Services and implementing Azure PaaS solutions.

Required Skills & Qualifications

• Bachelor’s or master’s in computer science, Engineering, or related field.
• 8+ years in Data Engineering or Data Architecture roles.
• Expertise in Python, SQL, Spark, Kafka, and Airflow.
• Proven experience with cloud platforms (AWS, Azure, or GCP).
• Hands-on experience with Snowflake, Databricks, or Delta Lake.
• Strong understanding of data modelling, governance, and lineage tools.
• Excellent problem-solving, leadership, and communication skills.

Good to Have Skills

• C#, .NET Core, Azure PaaS, Problem-Solving, REST API Development, EF Core, Leadership, Agile Practices.
• Blazor UI, Azure IPaaS (Azure Integration Services), Design Patterns, and Cloud Security best practices.