Sr. Azure Databricks Engineer

Persistent Systems; Bengaluru; Hybrid; INR 2,800,000 - 4,500,000; Full time

Job summary

Persistent Systems is seeking an experienced Senior Azure Data Engineer to design and deliver scalable data solutions on Azure, focusing on Databricks, Snowflake, and ETL/ELT pipelines. You will work with cross-functional teams to ensure data quality, efficiency, and scalable analytics across business use cases.

The role requires strong SQL, Python, and cloud data services expertise, with a hands-on approach to building high-performance data pipelines and data warehouses in Azure.

Qualifications

• 8–16 years of data engineering experience.
• Hands-on Azure Databricks and PySpark experience.
• Extensive Snowflake experience (data modeling, ETL/ELT).
• Strong SQL and Python skills.
• Experience with Azure Data Factory, ADLS, Synapse.
• Experience with version control and CI/CD (Git, Azure DevOps).

Responsibilities

• Design and develop scalable data pipelines using Azure Databricks (PySpark).
• Build and maintain robust ETL/ELT workflows for large-scale data processing.
• Develop and optimize SQL queries in Snowflake for analytics and reporting.
• Integrate data pipelines using Azure services such as ADF, ADLS, and Synapse.
• Design and manage Snowflake data warehouse structures.
• Perform performance tuning of Spark jobs and Snowflake workloads.
• Implement data quality, validation, governance, and monitoring mechanisms.
• Collaborate with business stakeholders to gather requirements and deliver data solutions.
• Troubleshoot and resolve data pipeline and production issues.

Skills

Azure Databricks

PySpark

Snowflake

SQL

Python

ADF/ADLS/Synapse

CI/CD

Data Warehousing

Tools

Git

Azure DevOps

Job description

We are looking for an experienced Senior Azure Data Engineer with strong expertise in Databricks, Snowflake, and ETL/ELT pipeline development. The candidate will be responsible for building scalable, high-performance data solutions on Azure to support analytics and business use cases. The ideal candidate will work closely with cross-functional teams to design and deliver high-performance, reliable, and data-driven solutions, while ensuring data quality, efficiency, and scalability within the Azure ecosystem.

• Experience: 8 to 16 Years
• Job Type: Full-Time Employment

What You'll Do:

• Design and develop scalable data pipelines using Azure Databricks (PySpark)
• Build and maintain robust ETL/ELT workflows for large-scale data processing
• Develop and optimize complex SQL queries in Snowflake for analytics and reporting
• Integrate data pipelines using Azure services such as ADF, ADLS, and Synapse
• Design and manage Snowflake data warehouse structures (tables, views, pipelines)
• Perform performance tuning of Spark jobs and Snowflake workloads
• Implement data quality, validation, governance, and monitoring mechanisms
• Collaborate with business stakeholders to gather requirements and deliver data solutions
• Troubleshoot and resolve data pipeline and production issues

Expertise You'll Bring:

• 8–12 years of experience in Data Engineering
• Strong hands-on experience with Azure Databricks and PySpark
• Extensive experience with Snowflake (data modeling, optimization, ETL/ELT pipelines)
• Strong proficiency in SQL and Python
• Experience with Azure services: Data Factory (ADF), Data Lake (ADLS), Synapse
• Solid understanding of data warehousing, ETL processes, and pipeline architecture
• Experience with version control and CI/CD tools (Git, Azure DevOps)
• Competitive salary and benefits package
• Culture focused on talent development with quarterly growth opportunities and company-sponsored higher education and certifications
• Opportunity to work with cutting-edge technologies
• Employee engagement initiatives such as project parties, flexible work hours, and Long Service awards
• Insurance coverage: group term life, personal accident, and Mediclaim hospitalization for self, spouse, two children, and parents

Values-Driven, People-Centric & Inclusive Work Environment:

Persistent is dedicated to fostering diversity and inclusion in the workplace. We invite applications from all qualified individuals, including those with disabilities, and regardless of gender or gender preference. We welcome diverse candidates from all backgrounds.

• We support hybrid work and flexible hours to fit diverse lifestyles.
• Our office is accessibility-friendly, with ergonomic setups and assistive technologies to support employees with physical disabilities.
• If you are a person with disabilities and have specific requirements, please inform us during the application process or at any time during your employment

Let’s unleash your full potential at Persistent - persistent.com/careers

"Persistent is an Equal Opportunity Employer and prohibits discrimination and harassment of any kind."