Lead Azure Data Engineer

KPI Partners; Bengaluru; On-site; INR 1,400,000 - 2,200,000; Full time

Job summary

KPI Partners is seeking an Azure Data Engineer to design and implement scalable data solutions in a hybrid setup across Hyderabad, Bengaluru, and Pune. You will work with PySpark, Databricks, SQL, and Azure data services to deliver robust data pipelines and analytics for global clients.

The ideal candidate has 4–8 years of hands-on data engineering experience and strong communication skills to engage with customers and stakeholders throughout the project lifecycle.

Qualifications

• 7 years of hands-on experience in Data Engineering.
• Strong proficiency in Python and Spark (PySpark).
• Expertise in writing efficient and complex SQL queries.
• Deep knowledge of Databricks and Azure data services.
• Good understanding of Azure Data Factory, Data Lake and related tools.
• Excellent communication skills; customer-facing roles.
• Ability to work in a collaborative, fast-paced environment.

Responsibilities

• Design, build, and maintain scalable data pipelines using PySpark and Databricks.
• Develop and optimize complex SQL queries for data transformation and analysis.
• Implement end-to-end data workflows in the Azure ecosystem (Data Factory, Data Lake, etc.).
• Collaborate with business stakeholders and cross-functional teams to understand data needs.
• Ensure high data quality, performance, and reliability in production environments.
• Participate in client-facing discussions and deliver technical solutions effectively.

Skills

Data Engineering

Python

Spark (PySpark)

SQL

Databricks

Azure Data Factory

Azure Data Lake

Customer-facing communication

Collaboration in teams

Tools

Databricks

Azure Synapse

Job description

Job Title: Azure Data Engineer

Experience: 4 to 8 Years Work Location: Hyderabad, Bengaluru, Pune Work Mode: Hybrid (2 days/week from office any of the above locations)

Notice Period: Less than 30 days

About KPI Partners:

KPI Partners is a leading consulting firm specializing in Data, Analytics, and Cloud solutions. We help global enterprises transform their data into actionable insights through modern platforms and customer-centric strategies.

Job Summary:

We are hiring experienced Azure Data Engineers who are passionate about building scalable data solutions. Ideal candidates should have strong expertise in Python, Spark (PySpark), SQL, and Databricks, with excellent communication skills to work closely with our customers.

Key Responsibilities:

• Design, build, and maintain scalable data pipelines using PySpark and Databricks.
• Develop and optimize complex SQL queries for data transformation and analysis.
• Implement end-to-end data workflows in the Azure ecosystem (Data Factory, Data Lake, etc.).
• Collaborate with business stakeholders and cross-functional teams to understand data needs.
• Ensure high data quality, performance, and reliability in production environments.
• Participate in client-facing discussions and deliver technical solutions effectively.

Required Skills:

• 7 years of hands-on experience in Data Engineering.
• Strong proficiency in Python and Spark (PySpark).
• Expertise in writing efficient and complex SQL queries.
• Deep knowledge of Databricks and Azure data services.
• Good understanding of Azure Data Factory, Data Lake, and related tools.
• Excellent communication skills all roles are customer-facing.
• Ability to work in a collaborative, fast-paced environment.

Nice to Have:

• Experience with Delta Lake, Lakehouse architectures.
• Knowledge of Azure Synapse, Power BI, or real-time data processing.
• Azure certifications (e.g., DP-203) are a plus.

Why Join Us?

• Work with cutting-edge cloud and data technologies.
• Be part of a high-growth, customer-focused consulting firm.
• Opportunity to contribute to large-scale digital transformation projects.
• Flexible hybrid work culture with locations in Hyderabad, Bengaluru, and Pune.