Databricks Developer - Pyspark and ADF

Zorba Consulting; Hyderabad, Pune District; On-site; INR 1,200,000 - 1,800,000; Full time

Job summary

Zorba Consulting is seeking a data engineer with strong Databricks and SQL background to migrate Hadoop data to AWS using Databricks. The role requires substantial Azure Databricks and PySpark experience, plus Python scripting and Airflow DevOps capabilities.

Knowledge of Agile methods and healthcare domain is preferred. You will contribute to lakehouse architecture, Delta Lake implementations, and SQL-based data processing within a fast-paced team in India.

Qualifications

• Databricks, SQL with at least 1 project migrating data from Hadoop to AWS using Databricks.
• 4+ years in Azure Databricks with PySpark stack.
• 1–2 years Databricks lakehouse and Delta Lake experience.
• 3+ years SQL scripting and stored procedures in Teradata/MS SQL Server.
• 3+ years Python scripting.
• Airflow DevOps experience.
• Agile knowledge and Healthcare domain knowledge.

Skills

Databricks

Azure Databricks

Pyspark

SQL scripting

Python scripting

Airflow

Agile knowledge

Healthcare domain knowledge

Tools

Databricks

SQL

Delta Lake

Hadoop

Job description

• Databricks, SQL At least 1 project experience in migrating data from HADOOP Bigdata into AWS using Databricks
• At least 4 + years strong experience in Azure Databricks, Pyspark stack
• At least 1-2 years experience working in Databricks lakehouse architecture and Deltalake
• At least 3 years strong experience in SQL scripting and experience in writing stored procedures in one of the RDBMS (teradata/MS SQL Server)
• At least 3+ years strong experience in writing Python scripts
• Experience in Data orchestration using Airflow DevOps
• Knowledge, Agile Knowledge, Healthcare Domain Knowledge.

Location - Delhi, Hyderabad, Pune,Kolkata,Bangalore,Chennai,Mumbai.