Senior Azure Databricks & Snowflake Data Engineer
Zorba AI
Get more replies from employers
Send a job-specific resume in minutes.
Job summary
Zorba AI seeks a highly skilled Azure Databricks & Snowflake Data Engineer in Chennai to design and maintain scalable data pipelines. You will work with Spark, PySpark, Python/Scala, and Snowflake, migrating workloads and building robust data processing frameworks across the SDLC.
Ideal candidates will have 5+ years of experience in Azure Databricks and Snowflake development, with strong SQL skills and familiarity with CI/CD and AI-assisted coding tools.
Qualifications
- 5+ years of hands-on Azure Databricks experience.
- 5+ years in Spark / PySpark.
- 5+ years programming in Python and/or Scala.
- 5+ years on Azure cloud platform.
- 3+ years Snowflake development experience.
- Strong SQL and database concepts (5+ years).
- CI/CD, Git, and GitHub Workflows experience.
- Strong software development and coding skills.
- 5+ years SDLC experience.
Responsibilities
- Design, develop, and optimize data engineering solutions using Azure Databricks, Spark, PySpark, Python/Scala.
- Build and optimize large-scale ETL/ELT pipelines and migrate workloads to Azure Databricks and Snowflake.
- Develop robust data transformation, ingestion, validation, and processing frameworks.
- Work with Snowflake for loading, transformation, optimization, and integration.
- Collaborate with architects, analysts, and business teams to deliver scalable solutions.
- Implement CI/CD via Git, GitHub Workflows, and Azure DevOps.
Skills
Education
Tools
Job description
We are looking for a highly skilled Azure Databricks & Snowflake Data Engineer with strong experience in Spark, PySpark, Python/Scala, Azure Cloud, SQL, ETL/ELT pipelines, and Snowflake development. The ideal candidate should have hands‑on experience in building and migrating enterprise data pipelines and should be comfortable working across the complete Software Development Life Cycle. Exposure to Snowflake Cortex/CoCo, Databricks Genie, and AI-assisted coding tools such as GitHub Copilot is highly desirable. Candidates with strong core engineering skills and a willingness to learn emerging AI capabilities will also be considered.
- Design, develop, optimize, and maintain scalable data engineering solutions using Azure Databricks, Apache Spark, PySpark, Python, and/or Scala.
- Develop and optimize complex ETL/ELT data pipelines for large-scale data processing.
- Perform migration of existing ETL/ELT pipelines and data workloads to Azure Databricks and Snowflake.
- Develop robust data transformation, ingestion, validation, and processing frameworks.
- Work extensively with Snowflake for data loading, transformation, optimization, and integration.
- Collaborate with data architects, developers, analysts, and business teams to understand requirements and deliver scalable solutions.
- Implement CI/CD pipelines and automated deployment processes using Git, GitHub, GitHub Workflows, and Azure DevOps.
- Follow software engineering best practices including version control, code reviews, unit testing, debugging, and documentation.
- Work with workflow/orchestration technologies such as Azure Data Factory (ADF), Azure Synapse, and Airflow.
- Participate in architecture discussions and contribute to improving data platform performance, reliability, and scalability.
- Utilize Terraform and Infrastructure-as-Code concepts where applicable.
- Explore and leverage AI-assisted development tools such as GitHub Copilot and Databricks Genie to improve engineering productivity.
- Gain exposure to and contribute to solutions using Snowflake Cortex/CoCo and other AI capabilities.
- Troubleshoot production issues and provide root‑cause analysis and permanent fixes.
- Participate in all phases of the Software Development Life Cycle (SDLC).
- 5+ years of hands‑on experience with Azure Databricks.
- 5+ years of experience with Apache Spark / PySpark.
- 5+ years of programming experience using Python and/or Scala.
- 5+ years of experience working on the Azure Cloud Platform.
- 3+ years of hands‑on Snowflake development experience.
- Strong knowledge of SQL and database concepts with 5+ years of experience.
- Strong experience in ETL/ELT pipeline development and migration.
- Hands‑on experience with CI/CD, Git, and GitHub Workflows.
- Strong software development and coding skills.
- 5+ years of complete SDLC experience.
- Knowledge or hands‑on experience with Snowflake Cortex / CoCo.
- Exposure to Databricks Genie.
- Experience using GitHub Copilot or other AI-assisted coding tools.
- Knowledge of Azure DevOps.
- Knowledge of Terraform / Infrastructure as Code.
- Experience with Azure Data Factory (ADF).
- Exposure to Azure Synapse Analytics.
- Exposure to Apache Airflow.
- Experience with Delta Lake and modern data lakehouse architecture.
- Understanding of data engineering best practices, performance tuning, and cloud-native architectures.
- Strong programming and problem‑solving skills.
- Excellent understanding of data structures, SQL, databases, and data processing concepts.
- Ability to develop clean, maintainable, reusable, and production‑ready code.
- Strong analytical and debugging skills.
- Ability to work independently as well as collaboratively in a distributed team.
- Good communication and stakeholder‑management skills.
- Strong willingness to learn emerging technologies, particularly Generative AI and AI-assisted development.
Bachelor’s or Master’s degree in Computer Science, Information Technology, Engineering, or a related discipline.
Evaluation / Skill Weightage
Evaluation AreaWeightageAzure Databricks / Spark / PySpark25%Python / Scala & Coding Skills15%Snowflake Development15%Azure Cloud & Data Engineering10%ETL/ELT & Pipeline Migration10%SQL / Database Expertise10%CI/CD, Git, GitHub & DevOps5%ADF / Synapse / Airflow / Terraform5%Snowflake Cortex/CoCo, Databricks Genie & AI-assisted Coding5%Total100%
Azure Databricks, Databricks, Apache Spark, PySpark, Python, Scala, Azure, Snowflake, Snowflake Cortex, CoCo, Databricks Genie, GitHub Copilot, ETL, ELT, Data Migration, SQL, Azure Data Factory, ADF, Azure Synapse, Airflow, Azure DevOps, Terraform, CI/CD, Git, GitHub Actions, Data Engineering, SDLC.
A candidate with 5+ years of experience in Azure Databricks, Spark/PySpark, Python/Scala, Azure Cloud, SQL, and end‑to‑end data engineering, along with 3+ years of Snowflake development experience. Strong ETL/ELT migration and CI/CD experience is essential. Exposure to Snowflake Cortex/CoCo, Databricks Genie, and GitHub Copilot will be an added advantage.
Skills: snowflake,azure,pyspark,sql
'/%3e%3cpath%20d='M101.744%2012.4726L102.894%2016.1947L99.0021%2013.2201L101.744%2012.4726ZM105.3%208.40937H100.488L99.0026%203.59473L97.5116%208.41047L92.7%208.40385L96.5965%2011.3834L95.1055%2016.1942L99.0021%2013.2196L101.408%2011.3834L105.3%208.40992V8.40937Z'%20fill='white'/%3e%3cdefs%3e%3clinearGradient%20id='paint0_linear_5593_1381'%20x1='90'%20y1='9.44471'%20x2='99'%20y2='9.44471'%20gradientUnits='userSpaceOnUse'%3e%3cstop%20stop-color='%2300AD70'/%3e%3cstop%20offset='0.998594'%20stop-color='%2300B67A'/%3e%3cstop%20offset='1'%20stop-color='%23DCDCE6'/%3e%3c/linearGradient%3e%3c/defs%3e%3c/svg%3e){kind=small}
