Azure Data Engineer

KPI Partners; Bengaluru; On-site; INR 4,000,000 - 6,000,000; Full time

Job summary

KPI Partners is seeking an Azure Data Engineer with expertise in data engineering, cloud solutions, and experience with data pipeline development using Azure services. You will play a key role in designing and implementing scalable data solutions that drive insights and business decisions for our clients.

7+ years of experience required. Proficiency in ADF, SQL, PySpark, Python, and Databricks is mandatory.

Qualifications

• 7+ years of experience as a Data Engineer, with focus on Azure data services.
• Experience with ADF, PySpark, Python, and SQL.
• Proficiency in SQL and programming languages like Python or Scala.
• Familiarity with Azure Data Lake, Azure SQL Database, and Azure Synapse Analytics.
• Strong analytical and problem-solving skills with attention to detail; excellent communication and collaboration skills.

Responsibilities

• Design, develop, and maintain data pipelines.
• Collaborate with data scientists, analysts, and stakeholders to understand data requirements and deliver high-quality data solutions.
• Optimize data workflows for performance and scalability within the Azure environment.
• Ensure data quality, integrity, and security throughout the data lifecycle.
• Monitor and troubleshoot data processing issues to maintain smooth operations.
• Document processes, architectures, and data models for knowledge sharing and compliance.
• Stay updated with the latest trends and technologies in data engineering and cloud computing.

Skills

ADF

SQL

Pyspark

Python

Databricks

Education

Bachelor’s degree in Computer Science / Data Science / Engineering

Tools

Azure Data Lake

Azure SQL Database

Azure Synapse Analytics

Job description

KPI Partners is seeking a skilled Azure Data Engineer with expertise in data engineering, cloud solutions, and experience with data pipeline development using Azure services. You will play a key role in designing and implementing scalable data solutions that drive insights and business decisions for our clients.

Experience: 7+ years

Mandatory Skills: ADF, SQL, Pyspark, Python, Databricks

Key Responsibilities

• Design, develop, and maintain data pipelines.
• Collaborate with data scientists, analysts, and stakeholders to understand data requirements and deliver high-quality data solutions.
• Optimize data workflows for performance and scalability within the Azure environment.
• Ensure data quality, integrity, and security throughout the data lifecycle.
• Monitor and troubleshoot data processing issues to maintain smooth operations.
• Document processes, architectures, and data models for knowledge sharing and compliance.
• Stay updated with the latest trends and technologies in data engineering and cloud computing.

Qualifications:

• Bachelor’s degree in Computer Science, Data Science, Engineering, or a related field.
• Proven 7+ experience as a Data Engineer, with a focus on AFD, Pyspark, Python and SQL.
• Proficiency in SQL and experience with programming languages such as Python or Scala.
• Familiarity with Azure data services, including Azure Data Lake, Azure SQL Database, and Azure Synapse Analytics.
• Strong analytical and problem-solving skills with attention to detail.
• Excellent communication and collaboration skills to work effectively across teams.