AWS Databricks Architect

Persistent Systems; Pune District; Hybrid; INR 4,000,000 - 7,000,000; Full time

Benefits offered by this job

Hybrid work model

Company-sponsored higher education & 2

Long Service awards

Insurance coverage: group term life, 7

Job summary

Persistent Systems is seeking an experienced AWS Databricks Architect with 13-20 years of enterprise data platform experience. You will design Lakehouse architectures using Databricks, Delta Lake, Unity Catalog, and AWS services to deliver scalable and secure data solutions.

You will lead migrations from on-prem to AWS Databricks, define governance and security standards, and mentor data engineering teams. This role requires hands-on leadership and collaboration with stakeholders to align

Qualifications

• Senior-level expertise in designing Databricks Lakehouse on AWS.
• Experience with data migration from on-prem to cloud.
• Strong governance, security, and cost-optimization skills.
• Excellent communication with stakeholders.

Responsibilities

• Design enterprise-scale Lakehouse and data warehouse architectures on AWS using Databricks, Delta Lake, Unity Catalog and S3.
• Define target-state architecture covering workspaces, networking, storage, compute, governance, security, and DR.
• Architect secure integrations between Databricks and AWS services including IAM, Glue, Redshift, Lambda, Kinesis, RDS, DMS, Secrets Manager, KMS, CloudWatch, and VPC.
• Lead data platform migration and modernization initiatives from on-premises and legacy environments to AWS Databricks platforms.
• Define and govern Medallion Architecture, data models, data quality frameworks, metadata standards, lineage, and governance controls.
• Establish enterprise security standards including IAM roles, encryption, secrets management, least-privilege access controls, private networking, and auditability practices.
• Drive performance tuning and cost optimization across clusters, serverless compute, SQL warehouses, storage architectures, and platform workloads.
• Define CI/CD frameworks, Infrastructure as Code standards, release processes, testing strategies, monitoring frameworks, and operational support models.
• Collaborate with business stakeholders, product owners, infrastructure teams, security teams, and engineering leaders to translate requirements into scalable architecture solutions.
• Create architecture blueprints, reference architectures, technology roadmaps, governance standards, and technical decision records.
• Lead architecture reviews, solution governance councils, and technical design workshops.
• Provide technical leadership, mentoring, troubleshooting, and architectural guidance to data engineering and platform teams.
• Evaluate emerging technologies and recommend architectural improvements aligned with business objectives.
• Support platform operationalization, reliability engineering, scalability planning, and enterprise adoption strategies.

Skills

Databricks Lakehouse

Delta Lake

Unity Catalog

AWS data services

PySpark / Spark SQL

Data governance & security

Terraform / IaC

Job description

We are seeking an experienced AWS Databricks Architect with 13-20 years of experience in designing and implementing enterprise-scale cloud data platforms. The ideal candidate will possess deep expertise in Databricks Lakehouse architecture, Delta Lake, Unity Catalog, AWS data services, data modernization, governance, security, and large-scale migration programs. This role requires a strong architecture mindset combined with hands-on technical leadership to deliver scalable, secure, cost-optimized, and business-aligned cloud data solutions.

• Role: AWS Databricks Architect
• Location: All Persistent Locations
• Experience: 13 to 20 Years
• Job Type: Full-Time Employment

What You'll Do:

• Design enterprise-scale Lakehouse and Data Warehouse architectures using Databricks, Delta Lake, Unity Catalog, and Amazon S3.
• Define target-state architecture covering Databricks workspaces, networking, storage, compute, governance, security, disaster recovery, and environment strategy.
• Architect secure integrations between Databricks and AWS services including Amazon S3, IAM, Glue, Redshift, Lambda, Kinesis, RDS, DMS, Secrets Manager, KMS, CloudWatch, and VPC.
• Design and oversee batch and real-time data ingestion, transformation, and processing pipelines using PySpark, Spark SQL, Databricks Workflows, Structured Streaming, and Delta Live Tables.
• Lead data platform migration and modernization initiatives from on-premises and legacy environments to AWS Databricks platforms.
• Define and govern Medallion Architecture, data models, data quality frameworks, metadata standards, lineage, and governance controls.
• Establish enterprise security standards including IAM roles, encryption, secrets management, least-privilege access controls, private networking, and auditability practices.
• Drive performance tuning and cost optimization across clusters, serverless compute, SQL warehouses, storage architectures, and platform workloads.
• Define CI/CD frameworks, Infrastructure as Code standards, release processes, testing strategies, monitoring frameworks, and operational support models.
• Collaborate with business stakeholders, product owners, infrastructure teams, security teams, and engineering leaders to translate requirements into scalable architecture solutions.
• Create architecture blueprints, reference architectures, technology roadmaps, governance standards, and technical decision records.
• Lead architecture reviews, solution governance councils, and technical design workshops.
• Provide technical leadership, mentoring, troubleshooting, and architectural guidance to data engineering and platform teams.
• Evaluate emerging technologies and recommend architectural improvements aligned with business objectives.
• Support platform operationalization, reliability engineering, scalability planning, and enterprise adoption strategies.

Expertise You'll Bring:

• 13 to 20 years of overall IT experience, including significant experience in enterprise data architecture and solution architecture roles.
• Extensive hands‑on experience designing and implementing Databricks‑based data platforms on AWS.
• Deep expertise in Databricks Lakehouse Platform, Delta Lake, Unity Catalog, Databricks SQL, Workflows, cluster policies, and serverless computing.
• Strong AWS architecture experience covering Amazon S3, IAM, VPC, Glue, Redshift, Kinesis, Lambda, RDS, DMS, CloudWatch, KMS, and Secrets Manager.
• Advanced proficiency in PySpark, Apache Spark, Spark SQL, Python, SQL, and distributed data processing architectures.
• Strong experience designing enterprise data lakes, data warehouses, Lakehouse platforms, ETL/ELT frameworks, and streaming data ecosystems.
• Proven expertise in leading cloud migration, data modernization, and enterprise transformation initiatives.
• Deep understanding of cloud security, governance, compliance, networking, identity management, resilience, and data privacy principles.
• Experience implementing Infrastructure as Code using Terraform, AWS CloudFormation, Git, and automated deployment pipelines.
• Strong experience in performance tuning, scalability planning, workload optimization, reliability engineering, and cloud cost management.
• Expertise in designing data governance frameworks, metadata management, data lineage, cataloging, and data quality controls.
• Ability to communicate complex technical concepts effectively to business stakeholders, technical teams, and executive leadership.
• Strong leadership experience in architecture governance, solution reviews, technical mentoring, and enterprise platform strategy.
• Excellent stakeholder management, presentation, problem-solving, and decision‑making capabilities.
• Ability to balance architectural standards, business objectives, delivery timelines, and operational realities.
• Competitive salary and benefits package
• Culture focused on talent development with quarterly growth opportunities and company‑sponsored higher education and certifications
• Opportunity to work with cutting‑edge technologies
• Employee engagement initiatives such as project parties, flexible work hours, and Long Service awards
• Insurance coverage: group term life, personal accident, and Mediclaim hospitalization for self, spouse, two children, and parents

Values-Driven, People-Centric & Inclusive Work Environment:

Persistent is dedicated to fostering diversity and inclusion in the workplace. We invite applications from all qualified individuals, including those with disabilities, and regardless of gender or gender preference. We welcome diverse candidates from all backgrounds.

• We support hybrid work and flexible hours to fit diverse lifestyles.
• Our office is accessibility-friendly, with ergonomic setups and assistive technologies to support employees with physical disabilities.
• If you are a person with disabilities and have specific requirements, please inform us during the application process or at any time during your employment

Let's unleash your full potential at Persistent - persistent.com/careers

"Persistent is an Equal Opportunity Employer and prohibits discrimination and harassment of any kind."