Senior Azure Databricks & Snowflake Data Engineer

Zorba AI; Maharashtra; On-site; INR 350,000 - 700,000; Full time

Job summary

Zorba AI is seeking a highly skilled Azure Databricks & Snowflake Data Engineer in Maharashtra to design and optimize end-to-end data pipelines. You will work with Spark, PySpark, Python/Scala, SQL, and Snowflake to deliver scalable data solutions across the SDLC.

Exposure to AI-assisted coding tools and modern data engineering practices is valued. The role requires migration of pipelines and hands-on cloud experience in Azure.

Qualifications

• Bachelor’s or Master’s degree in Computer Science, Information Technology, Engineering, or a related discipline.
• 5+ years hands-on Azure Databricks, Spark/PySpark experience.
• 5+ years Python/Scala programming, 5+ years Azure Cloud experience.

Responsibilities

• Design, develop, and maintain scalable data engineering solutions using Azure Databricks, Spark, PySpark, Python, and/or Scala.
• Develop and optimize complex ETL/ELT pipelines for large-scale data processing.
• Migrate existing ETL/ELT pipelines and workloads to Azure Databricks and Snowflake.
• Develop robust data transformation, ingestion, validation, and processing frameworks.
• Collaborate with architects, developers, analysts, and business teams to deliver scalable solutions.
• Implement CI/CD pipelines and automated deployment using Git, GitHub Workflows, and Azure DevOps.

Skills

Azure Databricks

Apache Spark

PySpark

Python

Scala

Azure Cloud

SQL

ETL/ELT pipelines

Snowflake development

Education

Bachelor’s or Master’s degree in CS/IT/Engineering

Tools

CI/CD

GitHub Workflows

Git

Databricks Genie

Snowflake Cortex/CoCo

GitHub Copilot

Terraform

Job description

We are looking for a highly skilled Azure Databricks & Snowflake Data Engineer with strong experience in Spark, PySpark, Python/Scala, Azure Cloud, SQL, ETL/ELT pipelines, and Snowflake development. The ideal candidate should have hands-on experience in building and migrating enterprise data pipelines and should be comfortable working across the complete Software Development Life Cycle.

Exposure to Snowflake Cortex/CoCo, Databricks Genie, and AI‑assisted coding tools such as GitHub Copilot is highly desirable. Candidates with strong core engineering skills and a willingness to learn emerging AI capabilities will also be considered.

Key Responsibilities

• Design, develop, optimize, and maintain scalable data engineering solutions using Azure Databricks, Apache Spark, PySpark, Python, and/or Scala.
• Develop and optimize complex ETL/ELT data pipelines for large-scale data processing.
• Perform migration of existing ETL/ELT pipelines and data workloads to Azure Databricks and Snowflake.
• Develop robust data transformation, ingestion, validation, and processing frameworks.
• Work extensively with Snowflake for data loading, transformation, optimization, and integration.
• Collaborate with data architects, developers, analysts, and business teams to understand requirements and deliver scalable solutions.
• Implement CI/CD pipelines and automated deployment processes using Git, GitHub, GitHub Workflows, and Azure DevOps.
• Follow software engineering best practices including version control, code reviews, unit testing, debugging, and documentation.
• Work with workflow/orchestration technologies such as Azure Data Factory (ADF), Azure Synapse, and Airflow.
• Participate in architecture discussions and contribute to improving data platform performance, reliability, and scalability.
• Utilize Terraform and Infrastructure-as-Code concepts where applicable.
• Explore and leverage AI-assisted development tools such as GitHub Copilot and Databricks Genie to improve engineering productivity.
• Gain exposure to and contribute to solutions using Snowflake Cortex/CoCo and other AI capabilities.
• Troubleshoot production issues and provide root‑cause analysis and permanent fixes.
• Participate in all phases of the Software Development Life Cycle (SDLC).

Mandatory Technical Skills

• 5+ years of hands‑on experience with Azure Databricks.
• 5+ years of experience with Apache Spark / PySpark.
• 5+ years of programming experience using Python and/or Scala.
• 5+ years of experience working on the Azure Cloud Platform.
• 3+ years of hands‑on Snowflake development experience.
• Strong knowledge of SQL and database concepts with 5+ years of experience.
• Strong experience in ETL/ELT pipeline development and migration.
• Hands‑on experience with CI/CD, Git, and GitHub Workflows.
• Strong software development and coding skills.
• 5+ years of complete SDLC experience.

Good-to-Have Skills

• Knowledge or hands‑on experience with Snowflake Cortex / CoCo.
• Exposure to Databricks Genie.
• Experience using GitHub Copilot or other AI‑assisted coding tools.
• Knowledge of Azure DevOps.
• Knowledge of Terraform / Infrastructure as Code.
• Experience with Azure Data Factory (ADF).
• Exposure to Azure Synapse Analytics.
• Exposure to Apache Airflow.
• Experience with Delta Lake and modern data lakehouse architecture.
• Understanding of data engineering best practices, performance tuning, and cloud‑native architectures.

Core Competencies

• Strong programming and problem‑solving skills.
• Excellent understanding of data structures, SQL, databases, and data processing concepts.
• Ability to develop clean, maintainable, reusable, and production‑ready code.
• Strong analytical and debugging skills.
• Ability to work independently as well as collaboratively in a distributed team.
• Good communication and stakeholder‑management skills.
• Strong willingness to learn emerging technologies, particularly Generative AI and AI‑assisted development.

Education

Bachelor’s or Master’s degree in Computer Science, Information Technology, Engineering, or a related discipline.

Evaluation / Skill WeightageEvaluation AreaWeightage

Azure Databricks / Spark / PySpark25%Python / Scala & Coding Skills15%Snowflake Development15%Azure Cloud & Data Engineering10%ETL/ELT & Pipeline Migration10%SQL / Database Expertise10%CI/CD, Git, GitHub & DevOps5%ADF / Synapse / Airflow / Terraform5%Snowflake Cortex/CoCo, Databricks Genie & AI‑assisted Coding5%Total100%

Key Search Keywords

Azure Databricks, Databricks, Apache Spark, PySpark, Python, Scala, Azure, Snowflake, Snowflake Cortex, CoCo, Databricks Genie, GitHub Copilot, ETL, ELT, Data Migration, SQL, Azure Data Factory, ADF, Azure Synapse, Airflow, Azure DevOps, Terraform, CI/CD, Git, GitHub Actions, Data Engineering, SDLC.

Ideal Candidate Profile

A candidate with 5+ years of experience in Azure Databricks, Spark/PySpark, Python/Scala, Azure Cloud, SQL, and end-to-end data engineering, along with 3+ years of Snowflake development experience. Strong ETL/ELT migration and CI/CD experience is essential. Exposure to Snowflake Cortex/CoCo, Databricks Genie, and GitHub Copilot will be an added advantage.

Skills: snowflake,azure,pyspark,sql