Azure Data Engineer

UsefulBI; India; On-site; INR 800,000 - 1,500,000; Full time

Job summary

UsefulBI is seeking a skilled Data Engineer to join their team in India. The ideal candidate should have a strong background in Python, SQL, and tools such as PySpark and Databricks, with at least 4 years of experience in developing data pipelines.

The role involves designing effective data pipelines, managing compliance, and closely collaborating with data scientists for high data quality. Familiarity with Azure Log Analytics and Databases is essential.

Qualifications

• Minimum 4 years of experience in data engineering required.
• Excellent problem-solving skills and attention to detail.
• Strong programming skills in Python and SQL.

Responsibilities

• Design, build, and maintain efficient and reliable data pipelines.
• Develop and implement ETL processes to integrate data.
• Troubleshoot data-related issues in a timely manner.
• Manage data security, privacy, and compliance requirements.
• Monitor performance and capacity of data pipelines.

Skills

Python

SQL

PySpark

Databricks

Azure Log analytics

Datafactory

Data Shares

Azure Synapse

Education

Bachelor’s or Master’s degree in Computer Science

Tools

Linux

Command-line tools

Job description

JOB DESCRIPTION

Job Summary:

We are looking for a skilled Data Engineer to join our team. The ideal candidate will have a strong background in Python, SQL, and PySpark and Databricks with experience in developing and managing data pipelines and databases.

Experience Required- Minimum 4 years (Must), Please do not apply if you do not have relevant experience.

Responsibilities:

• Design, build, and maintain efficient and reliable data pipelines and databases.
• Develop and implement ETL processes to integrate data from various sources into a unified view.
• Work closely with data scientists and analysts to ensure the quality and accuracy of data.
• Troubleshoot data-related issues and provide solutions in a timely manner.
• Manage data security, privacy, and compliance requirements.
• Monitor performance and capacity of data pipelines and databases.
• Continuously improve data engineering processes and practices.
• Experience in Azure Log analytics, Data Shares, Data factory and Azure synapse is must.

Requirements:

Excellent problem-solving skills and attention to detail.

Bachelor’s or Master’s degree in Computer Science or a related field.

4+ years of experience in data engineering.

Strong programming skills in Python and SQL.

Experience with Spark and Databricks and related tools such as PySpark or Spark.

Experience in Azure Log analytics, Data Shares, Data factory and Azure synapse is must.

Familiarity with data modeling, database design, and data warehousing concepts.

Proficiency in Linux and command-line tools.