Databricks Architect

Persistent Systems Limited; Bengaluru; Hybrid; INR 4,500,000 - 7,500,000; Full time

Benefits offered by this job

Competitive salary and benefits

Culture focused on talent development

Cutting-edge technologies

Employee engagement initiatives

Insurance coverage: life, accident, to

Job summary

Persistent Systems Ltd. is seeking an experienced Databricks Architect to lead the design and implementation of enterprise-scale data and AI platforms on Databricks.

You will define data modernization strategies, set architecture standards, and drive cloud transformation initiatives while collaborating with business and technology stakeholders. The role focuses on building scalable data lakehouse solutions, governance, MLOps, and AI/ML workloads across clouds, with leadership responsibilities

Qualifications

• 12 to 15 years of overall IT experience with at least 8+ years in Data Engineering, Data Analytics, and Data Platform Architecture.
• Hands-on expertise with Databricks Lakehouse Platform.
• Deep knowledge of Delta Lake, Unity Catalog, Lakehouse architecture, and modern data platform design.
• Strong expertise in Apache Spark (PySpark/Spark SQL).
• Experience in large-scale cloud migration and data modernization initiatives.

Responsibilities

• Define enterprise data platform strategy and Databricks architecture roadmap.
• Design scalable Lakehouse architectures on Databricks across cloud providers (Azure/AWS/GCP).
• Establish architecture standards, governance, and best practices.
• Lead end-to-end data solutions—ingestion, transformation, storage, analytics, AI/ML, and reporting.
• Architect batch, streaming, and real-time data processing frameworks.
• Implement Medallion Architecture (Bronze/Silver/Gold) and data governance.

Skills

Databricks Lakehouse Platform

Apache Spark / PySpark / Spark SQL

Python

SQL

Scala

Medallion Architecture

Delta Lake

Unity Catalog

Data governance

MLOps

CI/CD

DevOps

Cloud platforms (Azure/AWS/GCP)

Kafka / Event streaming

Education

Bachelor’s or Master’s degree in Computer Science/IT/Engineering or related field

Tools

Kafka

Azure Event Hubs

Kinesis

Unity Catalog

Job description

We are an AI-led, platform-driven Digital Engineering and Enterprise Modernization partner, combining deep technical expertise and industry experience to help our clients anticipate what?s next. Our offerings and proven solutions create a unique competitive advantage for our clients by giving them the power to see beyond and rise above. We work with many industry-leading organizations across the world, including 20 Fortune 50 companies and 4 of the 5 top banks in both the US and India, and numerous innovators across the healthcare ecosystem.

We are seeking a highly experienced Databricks Architect to lead the design, architecture, and implementation of enterprise-scale data and AI platforms on Databricks. The ideal candidate will define data modernization strategies, establish architecture standards, drive cloud transformation initiatives, and collaborate with business and technology stakeholders to deliver scalable, secure, and high-performance data solutions.

• Role: Databricks Architect
• Experience: 12 to 15 Years
• Job Type: Full Time Employment

What You'll Do:

• Define enterprise data platform strategy and Databricks architecture roadmap.
• Design scalable Lakehouse architectures using Databricks on Azure, AWS, or GCP.
• Establish architecture standards, governance frameworks, and best practices.
• Drive enterprise-wide data modernization and cloud migration initiatives.
• Architect end-to-end data solutions covering data ingestion, transformation, storage, analytics, AI/ML, and reporting.
• Design batch, streaming, and real-time data processing frameworks.
• Lead implementation of Medallion Architecture, Delta Lake, Unity Catalog, and data governance solutions.
• Ensure solutions meet scalability, reliability, security, and performance requirements.
• Architect cloud-native data platforms integrated with enterprise ecosystems.
• Define infrastructure patterns, CI/CD pipelines, DevOps, and DataOps practices.
• Optimize platform performance, cost management, and operational efficiency.
• Implement monitoring, observability, disaster recovery, and business continuity strategies.
• Enable AI/ML and Generative AI workloads on Databricks platforms.
• Design MLOps frameworks and model lifecycle management processes.
• Drive adoption of advanced analytics, predictive modeling, and AI-powered solutions.
• Collaborate with Data Scientists and Engineering teams to operationalize production-grade AI solutions.
• Provide technical leadership and architecture governance across multiple programs.
• Mentor architects, technical leads, and engineering teams.
• Conduct architecture reviews and ensure compliance with enterprise standards.
• Engage with executive stakeholders to align technology strategies with business objectives.
• Support customer engagement, solution consulting, and pre-sales initiatives.

Expertise You'll Bring:

• 12 to 15 years of overall IT experience with at least 8+ years in Data Engineering, Data Analytics, and Data Platform Architecture.
• Extensive hands‑on expertise with Databricks Lakehouse Platform.
• Deep knowledge of Delta Lake, Unity Catalog, Lakehouse architecture, and modern data platform design.
• Strong expertise in Apache Spark, including PySpark and Spark SQL.
• Advanced proficiency in Python, SQL, and Scala.
• Strong understanding of Data Warehousing, Lakehouse architectures, and enterprise analytics ecosystems.
• Experience driving large-scale cloud migration and data modernization initiatives.
• Expertise implementing Medallion Architecture (Bronze, Silver, Gold layers) for enterprise data platforms.
• Experience architecting batch, streaming, and real-time data processing solutions.
• Hands‑on experience with Kafka, Azure Event Hubs, Kinesis, or similar streaming technologies.
• Strong understanding of Data Governance, Data Quality, Metadata Management, and Security frameworks.
• Experience implementing Unity Catalog, access controls, lineage management, and governance best practices.
• Knowledge of MLOps, Model Lifecycle Management, Feature Stores, and AI platform architecture.
• Experience enabling Generative AI, Large Language Models (LLMs), Retrieval‑Augmented Generation (RAG), and advanced analytics workloads on Databricks.
• Strong familiarity with CI/CD pipelines, Infrastructure as Code, DevOps, and DataOps methodologies.
• Experience designing highly available, secure, scalable, and cost‑optimized cloud platforms.
• Expertise in observability, monitoring, logging, disaster recovery, and operational excellence practices.
• Strong enterprise architecture and strategic planning capabilities.
• Excellent stakeholder management and executive communication skills.
• Proven ability to lead architecture reviews, technical governance processes, and enterprise transformation initiatives.
• Experience mentoring architects, technical leads, and engineering teams.
• Strong consulting, customer engagement, and pre‑sales support experience.
• Ability to align technology strategy with business objectives and organizational goals.
• Databricks Certified Solutions Architect certification is highly desirable.
• Databricks Data Engineer Professional certification is preferred.
• Azure Solutions Architect Expert or AWS Solutions Architect Professional certification is advantageous.
• Bachelor?s or Master?s degree in Computer Science, Information Technology, Engineering, or a related field.
• Strong passion for innovation, cloud transformation, AI‑powered data platforms, and continuous learning.
• Competitive salary and benefits package
• Culture focused on talent development with quarterly growth opportunities and company-sponsored higher education and certifications
• Opportunity to work with cutting‑edge technologies
• Employee engagement initiatives such as project parties, flexible work hours, and Long Service awards
• Insurance coverage: group term life, personal accident, and Mediclaim hospitalization for self, spouse, two children, and parents

Values‑Driven, People‑Centric & Inclusive Work Environment:

Persistent is dedicated to fostering diversity and inclusion in the workplace. We invite applications from all qualified individuals, including those with disabilities, and regardless of gender or gender preference. We welcome diverse candidates from all backgrounds.

• We support hybrid work and flexible hours to fit diverse lifestyles.
• Our office is accessibility-friendly, with ergonomic setups and assistive technologies to support employees with physical disabilities.
• If you are a person with disabilities and have specific requirements, please inform us during the application process or at any time during your employment

?Persistent is an Equal Opportunity Employer and prohibits discrimination and harassment of any kind.?