Senior Databricks Developer

Persistent Systems Limited

Pune

Hybrid

INR 3,000,000 - 6,000,000

Full time

14 hours ago
Be an early applicant

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Benefits offered by this job

Group term life
Mediclaim hospitalization
Long Service awards
Education reimbursement
Flexible work hours
Hybrid work

Job summary

Persistent Systems Limited is seeking a Senior Databricks Developer with 7 to 12 years of experience in Data Engineering, Analytics, and Modern Data Platforms. You will design and optimize large-scale data pipelines, build lakehouse architectures, and deliver enterprise-grade data solutions across cloud environments.

You will work across multiple cloud ecosystems (Azure/AWS/GCP) and collaborate with data architects, engineers, and business stakeholders to drive platform modernization and enforce

Qualifications

  • 7 to 12 years of experience in Data Engineering, Analytics, and Modern Data Platforms.
  • Hands-on Databricks, PySpark, Spark SQL, Delta Lake expertise.
  • Experience delivering end-to-end Databricks implementations in enterprise environments.
  • Strong understanding of distributed computing, Spark architecture, and performance tuning.
  • Knowledge of CI/CD, Git-based deployment, and cloud-native data services.

Responsibilities

  • Design, develop, and maintain scalable ETL/ELT pipelines using Databricks, PySpark, SQL, and notebooks.
  • Build data workflows across Bronze, Silver, and Gold layers with Delta Lake.
  • Develop and optimize batch and real-time streaming pipelines using Structured Streaming.
  • Tune Spark performance, manage clusters, and optimize costs for platform efficiency.
  • Implement data quality, monitoring, and governance across data pipelines.
  • Integrate Databricks with Azure/AWS/GCP services and collaborate with stakeholders.
  • Troubleshoot data pipelines, clusters, jobs, and runtime environments; ensure security and compliance.
  • Adopt CI/CD practices with Git and deployment automation; mentor junior engineers.

Skills

Databricks
PySpark
Spark SQL
Delta Lake
SQL
CI/CD
Git
Cloud platforms

Tools

Notebooks
Azure
AWS
GCP

Job description

We are an AI-led, platform-driven Digital Engineering and Enterprise Modernization partner, combining deep technical expertise and industry experience to help our clients anticipate what’s next. Our offerings and proven solutions create a unique competitive advantage for our clients by giving them the power to see beyond and rise above. We work with many industry-leading organizations across the world, including 18 of the Fortune 50 companies, 7 of the 10 top US banks, 9 of the 10 technology companies, and numerous innovators across the healthcare ecosystem.

Our disruptor’s mindset, commitment to client success, and agility to thrive in the dynamic environment have enabled us to sustain our growth momentum. Persistent has been recognized across top industry platforms for innovation, leadership, and inclusion. We have delivered 26sequential quarters of growth with $487.6M in Q2 FY27 revenue, up 7.8% Q-o-Q and 20.0% Y-o-Y. Our 29,500+global team members, located in 21 countries, have been instrumental in helping the market leaders transform their industries. We received five awards at the ISG (Information Services Group) Provider Lens™ Awards 2026 and, for the second year in a row, were awarded the Growth Honor of the Year at the Everest Group Elevate 2026.

About Position:

We are seeking an experienced Senior Databricks Developer with 7 to 12 years of experience in Data Engineering, Analytics, and Modern Data Platforms. The ideal candidate will have strong hands-on expertise in Databricks, PySpark, Delta Lake, and cloud-based data ecosystems. This role involves designing and optimizing large-scale data pipelines, driving performance improvements, implementing scalable Lakehouse architectures, and delivering enterprise-grade data solutions across cloud environments.

  • Role: Senior Databricks Developer
  • Location: All Persistent Locations
  • Experience: 7 to 12 Years
  • Job Type: Full-Time Employment
What You'll Do:
  • Design, develop, and maintain scalable ETL/ELT pipelines using Databricks, PySpark, SQL, and Notebooks.
  • Build and manage data processing workflows across Bronze, Silver, and Gold layers using Delta Lake architecture.
  • Develop and optimize batch and real-time streaming data pipelines using Structured Streaming.
  • Perform Spark performance tuning, cluster optimization, and cost optimization to improve platform efficiency.
  • Implement robust data quality, validation, and monitoring frameworks across data pipelines.
  • Integrate Databricks solutions with cloud-native services across Azure, AWS, or GCP environments.
  • Collaborate with data architects, analysts, business stakeholders, and engineering teams to deliver scalable data solutions.
  • Troubleshoot and resolve issues related to data pipelines, clusters, jobs, and runtime environments.
  • Implement CI/CD best practices using Git and deployment automation tools.
  • Ensure adherence to security, governance, and compliance standards across data platforms.
  • Drive platform modernization initiatives and recommend best practices for scalable data engineering.
Expertise You'll Bring:
  • 7 to 12 years of experience in Data Engineering, Big Data, Analytics, and Modern Data Platforms.
  • Strong hands-on expertise in Databricks, PySpark, Spark SQL, and Delta Lake.
  • Experience delivering multiple end-to-end Databricks implementations in enterprise environments.
  • Deep understanding of distributed computing concepts and Spark architecture.
  • Strong expertise in cluster management, job orchestration, workload optimization, and performance tuning.
  • Experience designing and supporting large-scale batch and streaming data processing solutions.
  • Proficiency in SQL and data modeling concepts.
  • Strong knowledge of Data Lake, Lakehouse, and modern Data Warehouse architectures.
  • Experience with Azure, AWS, or GCP, including cloud-native data services.
  • Knowledge of CI/CD pipelines, Git-based version control, and automated deployment practices.
  • Strong troubleshooting and problem-solving skills with the ability to resolve complex production issues.
  • Excellent communication, stakeholder management, and collaboration skills.
  • Ability to mentor junior engineers and provide technical leadership across projects.
  • Competitive salary and benefits package
  • Culture focused on talent development with quarterly growth opportunities and company-sponsored higher education and certifications
  • Opportunity to work with cutting-edge technologies
  • Employee engagement initiatives such as project parties, flexible work hours, and Long Service awards
  • Insurance coverage: group term life, personal accident, and Mediclaim hospitalization for self, spouse, two children, and parents
Values-Driven, People-Centric & Inclusive Work Environment:

Persistent is dedicated to fostering diversity and inclusion in the workplace. We invite applications from all qualified individuals, including those with disabilities, and regardless of gender or gender preference. We welcome diverse candidates from all backgrounds.

  • We support hybrid work and flexible hours to fit diverse lifestyles.
  • Our office is accessibility-friendly, with ergonomic setups and assistive technologies to support employees with physical disabilities.
  • If you are a person with disabilities and have specific requirements, please inform us during the application process or at any time during your employment

"Persistent is an Equal Opportunity Employer and prohibits discrimination and harassment of any kind."

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Senior Azure Data Engineer
Senior Azure Data Engineer

Persistent Systems Limited • Bengaluru

Hybrid
INR 3,000,000 - 5,000,000
Hybrid work
Flexible hours
Long Service awards
+1
Lead Databricks Data Engineer
Lead Databricks Data Engineer

Persistent Systems • Bengaluru

Hybrid
INR 3,500,000 - 6,000,000
Hybrid work model
Medical insurance
Learning @ Work
+2
Azure Data Engineer
Azure Data Engineer

Persistent Systems Limited • Pune

Hybrid
INR 1,500,000 - 2,800,000
Competitive salary and benefits
Talent development with education/cert
Flexible work hours
+2
Senior Azure Data Engineer
Senior Azure Data Engineer

Persistent Systems • Pune District

Hybrid
INR 3,500,000 - 5,500,000
Competitive salary
Education stipend
Hybrid work model
+3
Snowflake Data Engineer
Snowflake Data Engineer

Persistent Systems Limited • Pune

Hybrid
INR 2,800,000 - 4,400,000
Competitive compensation & benefits
Talent development & education Spons
Cutting-edge technologies
+4
Senior Azure Data Engineer
Senior Azure Data Engineer

Persistent Systems • Pune District

On-site
INR 2,400,000 - 4,000,000
Hybrid work model
Flexible work hours
Long Service awards
+1
Azure Data Engineer
Azure Data Engineer

Persistent Systems • Mumbai

Hybrid
INR 1,200,000 - 2,000,000
Higher education sponsorship
Flexible work hours
Long Service awards
+1
Senior Azure Data Engineer ? Databricks, Kafka & Spark
Senior Azure Data Engineer ? Databricks, Kafka & Spark

Persistent Systems • Pune District

Hybrid
INR 1,200,000 - 1,800,000
Competitive salary and benefits
Culture focused on talent development
Flexible work hours
+2
Senior Azure Data Engineer
Senior Azure Data Engineer

Mployee.me • Pune District

Hybrid
INR 2,000,000 - 4,000,000
Competitive salary
Growth opportunities
Cutting-edge technology
+3
Senior Azure Data Engineer
Senior Azure Data Engineer

Persistent Systems Limited • Pune

Hybrid
INR 1,500,000 - 2,100,000
Hybrid work
Education sponsorship
Long Service awards
+1