Databricks Architect

Persistent Systems; Pune District; Hybrid; INR 4,000,000 - 7,000,000; Full time

Benefits offered by this job

Hybrid work model

Company-sponsored higher education and

certifications

Job summary

Persistent Systems is seeking an experienced Databricks Architect to build and optimize scalable data pipelines in Pune. The role requires 12+ years of experience and full-time employment.

You will develop ETL/ELT pipelines, implement Delta Lake architectures, and lead large-scale data platform initiatives. You will work with Databricks, PySpark, Spark SQL, and Unity Catalog, leveraging cloud platforms (Azure/AWS/GCP) while adopting CI/CD and governance practices.

Qualifications

• Extensive experience with Databricks and Spark-based data pipelines.
• Strong governance and Lakehouse data modeling proficiency.
• Expertise in CI/CD, Git workflows, and DevOps within data platforms.
• Proven ability to lead large-scale data platform implementations.
• Hands-on with cloud platforms (Azure/AWS/GCP).

Responsibilities

• Develop and maintain ETL/ELT pipelines using Databricks (PySpark, SQL, Notebooks).
• Define and implement Medallion Architecture (Bronze/Silver/Gold) using Delta Lake.
• Build high-performance ETL/ELT pipelines using PySpark, Spark SQL, Delta Lake.
• Architect and optimize batch and streaming data pipelines.
• Perform advanced Spark optimization, cluster sizing, and cost management.
• Implement CI/CD, version control, and DevOps best practices within Databricks.

Skills

Databricks Lakehouse Platform

PySpark

Spark SQL

Delta Lake

Structured Streaming

Unity Catalog

Cloud platforms (Azure/AWS/GCP)

Distributed computing

Data warehousing concepts

Data modeling (dimensional, Lakehouse)

CI/CD pipelines

Git

Orchestration tools

Leading large-scale data platform Imps

Tools

Databricks

Delta Lake

Git

Orchestration tools

Job description

We are looking for an experienced Databricks Architect who can build, optimize, and maintain scalable data pipelines on the Databricks platform.

• Role: Databricks Architect
• Location: Pune
• Experience: 12+ years
• Job Type: Full-Time Employment

What You'll Do:

• Develop and maintain ETL/ELT pipelines using Databricks (PySpark, SQL, Notebooks).
• Define and implement Medallion Architecture (Bronze/Silver/Gold) using Delta Lake.
• Build high-performance ETL/ELT pipelines using PySpark, Spark SQL, Delta Lake.
• Architect and optimize batch and streaming data pipelines.
• Perform advanced Spark optimization, cluster sizing, and cost management.
• Implement CI/CD, version control, and DevOps best practices within Databricks.

Expertise You'll Bring:

• Advanced expertise in:
• Strong understanding of:
• Databricks Lakehouse Platform
• PySpark & Spark SQL
• Delta Lake, Structured Streaming
• Unity Catalog & governance frameworks
• Strong experience with a major cloud platform (Azure/AWS/GCP).
• Distributed computing & Spark internals
• Data warehousing concepts
• Data modeling (dimensional, normalized, Lakehouse modeling)
• Experience in CI/CD pipelines, Git, orchestration tools.
• Experience in leading large‑scale data platform implementations.
• Competitive salary and benefits package
• Culture focused on talent development with quarterly growth opportunities and company-sponsored higher education and certifications
• Opportunity to work with cutting-edge technologies
• Employee engagement initiatives such as project parties, flexible work hours, and Long Service awards
• Insurance coverage: group term life, personal accident, and Mediclaim hospitalization for self, spouse, two children, and parents

Values-Driven, People-Centric & Inclusive Work Environment:

Persistent is dedicated to fostering diversity and inclusion in the workplace. We invite applications from all qualified individuals, including those with disabilities, and regardless of gender or gender preference. We welcome diverse candidates from all backgrounds.

• We support hybrid work and flexible hours to fit diverse lifestyles.
• Our office is accessibility-friendly, with ergonomic setups and assistive technologies to support employees with physical disabilities.
• If you are a person with disabilities and have specific requirements, please inform us during the application process or at any time during your employment

Let's unleash your full potential at Persistent - persistent.com/careers

“Persistent is an Equal Opportunity Employer and prohibits discrimination and harassment of any kind.”