Azure Databricks & Snowflake Data Engineer

Zorba Consulting; Mumbai, Delhi, Bengaluru; Hybrid; INR 1,200,000 - 2,400,000; Full time

Job summary

Zorba Consulting is seeking an Azure Databricks & Snowflake Data Engineer to design, build, migrate, and optimize enterprise-scale data pipelines. You will work with Spark, PySpark, Python/Scala, and SQL to develop robust ingestion, transformation, and processing frameworks across Snowflake and Azure Databricks.

The role spans the full SDLC with CI/CD adoption and collaboration across data teams. The candidate should have hands-on experience with Azure Data Factory, Airflow, and a modern cloud

Qualifications

• 5+ years hands-on Azure Databricks experience.
• 5+ years Apache Spark / PySpark experience.
• 5+ years Python and/or Scala programming.
• 5+ years on the Azure Cloud Platform.
• 3+ years Snowflake development experience.
• Strong SQL and database concepts with 5+ years.
• Experience in ETL/ELT pipeline development and migration.
• Hands-on CI/CD, Git and GitHub Workflows experience.
• Strong software development and coding skills.
• 5+ years of complete SDLC experience.

Responsibilities

• Design, develop, and optimize scalable data engineering solutions using Azure Databricks, Spark, PySpark, Python, and/or Scala.
• Develop and optimize complex ETL/ELT data pipelines for large-scale processing.
• Migrate existing ETL/ELT pipelines to Azure Databricks and Snowflake.
• Develop robust data transformation, ingestion, validation, and processing frameworks.
• Work with Snowflake for data loading, transformation, optimization, and integration.
• Collaborate with data architects, developers, analysts, and business teams to deliver scalable solutions.
• Implement CI/CD pipelines and automated deployment using Git, GitHub Workflows, and Azure DevOps.
• Follow software engineering best practices including version control, code reviews, unit testing, debugging, and documentation.
• Work with workflow/orchestration technologies such as ADF, Synapse, and Airflow.
• Participate in architecture discussions and improve data platform performance, reliability, and scalability.
• Utilize Terraform and IaC concepts where applicable.
• Explore AI-assisted development tools to improve engineering productivity.
• Gain exposure to Snowflake Cortex/CoCo and other AI capabilities.
• Troubleshoot production issues and provide root-cause analysis and fixes.
• Participate in all phases of the SDLC.

Skills

Strong programming skills

Data engineering

Analytical and debugging skills

Stakeholder communication

Tools

Azure Databricks

Apache Spark

PySpark

Python

Scala

Azure Cloud

SQL

ETL/ELT pipelines

Git

GitHub Workflows

Azure DevOps

Terraform

Airflow

Azure Data Factory

Azure Synapse

Databricks Genie

Snowflake

Snowflake Cortex/CoCo

Delta Lake

Job description

• We are looking for a highly skilled Azure Databricks & Snowflake Data Engineer with strong experience in Spark, PySpark, Python/Scala, Azure Cloud, SQL, ETL/ELT pipelines, and Snowflake development.
• The ideal candidate should have hands-on experience in building and migrating enterprise data pipelines and should be comfortable working across the complete Software Development Life Cycle.
• Exposure to Snowflake Cortex/CoCo, Databricks Genie, and AI-assisted coding tools such as GitHub Copilot is highly desirable.
• Candidates with strong core engineering skills and a willingness to learn emerging AI capabilities will also be considered.

Key Responsibilities

• Design, develop, optimize, and maintain scalable data engineering solutions using Azure Databricks, Apache Spark, PySpark, Python, and/or Scala.
• Develop and optimize complex ETL/ELT data pipelines for large-scale data processing.
• Perform migration of existing ETL/ELT pipelines and data workloads to Azure Databricks and Snowflake.
• Develop robust data transformation, ingestion, validation, and processing frameworks.
• Work extensively with Snowflake for data loading, transformation, optimization, and integration.
• Collaborate with data architects, developers, analysts, and business teams to understand requirements and deliver scalable solutions.
• Implement CI/CD pipelines and automated deployment processes using Git, GitHub, GitHub Workflows, and Azure DevOps.
• Follow software engineering best practices including version control, code reviews, unit testing, debugging, and documentation.
• Work with workflow/orchestration technologies such as Azure Data Factory (ADF), Azure Synapse, and Airflow.
• Participate in architecture discussions and contribute to improving data platform performance, reliability, and scalability.
• Utilize Terraform and Infrastructure-as-Code concepts where applicable.
• Explore and leverage AI-assisted development tools such as GitHub Copilot and Databricks Genie to improve engineering productivity.
• Gain exposure to and contribute to solutions using Snowflake Cortex/CoCo and other AI capabilities.
• Troubleshoot production issues and provide root-cause analysis and permanent fixes.
• Participate in all phases of the Software Development Life Cycle (SDLC).

Mandatory Technical Skills

• 5+ years of hands-on experience with Azure Databricks.
• 5+ years of experience with Apache Spark / PySpark.
• 5+ years of programming experience using Python and/or Scala.
• 5+ years of experience working on the Azure Cloud Platform.
• 3+ years of hands-on Snowflake development experience.
• Strong knowledge of SQL and database concepts with 5+ years of experience.
• Strong experience in ETL/ELT pipeline development and migration.
• Hands-on experience with CI/CD, Git, and GitHub Workflows.
• Strong software development and coding skills.
• 5+ years of complete SDLC experience.

Good-to-Have Skills

• Knowledge or hands-on experience with Snowflake Cortex / CoCo.
• Exposure to Databricks Genie.
• Experience using GitHub Copilot or other AI-assisted coding tools.
• Knowledge of Azure DevOps.
• Knowledge of Terraform / Infrastructure as Code.
• Experience with Azure Data Factory (ADF).
• Exposure to Azure Synapse Analytics.
• Exposure to Apache Airflow.
• Experience with Delta Lake and modern data lakehouse architecture.
• Understanding of data engineering best practices, performance tuning, and cloud-native architectures.
• Core Competencies Strong programming and problem-solving skills.
• Excellent understanding of data structures, SQL, databases, and data processing concepts.
• Ability to develop clean, maintainable, reusable, and production-ready code.
• Strong analytical and debugging skills.
• Ability to work independently as well as collaboratively in a distributed team.
• Good communication and stakeholder-management skills.
• Strong willingness to learn emerging technologies, particularly Generative AI and AI-assisted development.

Location - Pan India,Mumbai, Delhi / NCR, Bengaluru , Kolkata, Chennai, Hyderabad, Ahmedabad, Pune.