Azure Databricks Data Engineer|3 to 5 Years| Gyansys | Hybrid

PradeepIt Global Consulting Pvt Ltd.; Bengaluru; On-site; INR 1,000,000 - 1,500,000; Full time

Job summary

A consulting firm is seeking a Data Engineer in Bengaluru with 3 to 5 years of experience in Azure Databricks, PySpark, and SQL. The role involves designing ETL pipelines, optimizing SQL queries, and collaborating with teams on data analytics. Candidates should have strong analytical skills and the ability to manage large datasets effectively. The position offers a hybrid work model.

Qualifications

• 3 to 5 years of experience in Azure Databricks, PySpark, and SQL.
• Strong SQL skills: complex joins, functions, procedures, query tuning.
• Ability to handle large datasets efficiently.

Responsibilities

• Design, develop, and maintain ETL pipelines using Azure Databricks.
• Write complex SQL queries, performs SQL tuning for improved efficiency.
• Work closely with cross-functional teams to support data analytics initiatives.

Skills

Azure Databricks

PySpark

SQL

Tools

Azure Data Factory

Azure Data Lake Storage

Job description

Overview

Azure Databricks Data Engineer | 3 to 5 Years | Gyansys | Hybrid

About the job

Position: Data Engineer
Experience: 3 to 12 years
Primary Skills: Azure Databricks, PySpark, SQL
Secondary Skills: ADF (Azure Data Factory)
Location: Bengaluru/Hyderabad
Mode of Work: Hybrid
Notice Period: 0 to 30 Days

Key Responsibilities

ETL Development & Data Processing:

• Design, develop, and maintain ETL pipelines using Azure Databricks, PySpark, and ADF.
• Extract, transform, and load data from multiple sources into Azure Data Lake Storage.
• Optimize and fine-tune ETL workflows for performance and scalability.

SQL & Query Optimization:

• Write complex SQL queries, joins, subqueries, and functions for data processing.
• Perform SQL tuning and query optimization for improved efficiency.
• Work with large datasets and ensure optimal data handling.
• Work with batch and streaming data processing for real-time analytics (preferred).
• Ensure data security, compliance, and reliability in cloud-based systems.

Collaboration & Problem-Solving:

• Work closely with cross-functional teams to support data analytics and business intelligence initiatives.
• Identify and resolve data-related issues proactively.
• Provide insights and recommendations for improving ETL performance.

Required Skills & Qualifications

• 3 to 5 years of experience in Azure Databricks, PySpark, and SQL.
• Strong SQL skills: complex joins, functions, procedures, query tuning.
• Hands-on experience in ADF (Azure Data Factory) and Azure Data Lake Storage.
• Ability to handle large datasets efficiently.
• Experience in performance optimization of ETL workflows.
• Good understanding of batch and streaming data processing (preferred).
• Strong analytical and problem-solving skills.
• Ability to work independently and take ownership of tasks.