AWS Databricks Architect

Mployee.me; Pune District; On-site; INR 3,000,000 - 6,000,000; Full time

Benefits offered by this job

Competitive salary and benefits

Talent development culture with growth

Cutting-edge technologies

Employee engagement initiatives & long

Annual health check-ups

Insurance coverage for family

Job summary

Persistent is seeking an experienced AWS Databricks Architect to design and implement enterprise‑scale cloud data platforms. You will build Lakehouse architectures using Databricks, Delta Lake and Unity Catalog on AWS, and govern security, governance, and cost optimization.

The role requires 13–20 years of IT experience, strong leadership, and hands‑on architecture capabilities to drive migrations from on‑premises to AWS Databricks platforms.

Qualifications

• 13 to 20 years of overall IT experience, including significant experience in enterprise data architecture and solution architecture roles.
• Extensive hands-on experience designing and implementing Databricks-based data platforms on AWS.
• Deep expertise in Databricks Lakehouse Platform, Delta Lake, Unity Catalog, Databricks SQL, Workflows, cluster policies and serverless computing.
• Strong AWS architecture experience covering Amazon S3, IAM, VPC, Glue, Redshift, Kinesis, Lambda, RDS, DMS, CloudWatch, KMS and Secrets Manager.
• Advanced proficiency in PySpark, Apache Spark, Spark SQL, Python, SQL and distributed data processing architectures.
• Experience designing enterprise data lakes, data warehouses, Lakehouse platforms, ETL/ELT frameworks and streaming data ecosystems.

Responsibilities

• Design enterprise-scale Lakehouse and Data Warehouse architectures using Databricks, Delta Lake, Unity Catalog, and Amazon S3.
• Define target-state architecture covering Databricks workspaces, networking, storage, compute, governance, security, disaster recovery, and environment strategy.
• Architect secure integrations between Databricks and AWS services including S3, IAM, Glue, Redshift, Lambda, Kinesis, RDS, DMS, Secrets Manager, KMS, CloudWatch, and VPC.
• Design and oversee batch and real-time data ingestion, transformation and processing pipelines using PySpark, Spark SQL, Databricks Workflows, Structured Streaming, and Delta Live Tables.
• Lead data platform migration and modernization initiatives from on-premises and legacy environments to AWS Databricks platforms.
• Define and govern Medallion Architecture, data models, data quality frameworks, metadata standards, lineage and governance controls.
• Establish enterprise security standards including IAM roles, encryption, secrets management, least-privilege access controls, private networking and auditability practices.
• Drive performance tuning and cost optimization across clusters, serverless compute, SQL warehouses, storage architectures and platform workloads.
• Define CI/CD frameworks, Infrastructure as Code standards, release processes, testing strategies, monitoring frameworks and operational support models.
• Collaborate with business stakeholders, product owners, infrastructure teams, security teams and engineering leaders to translate requirements into scalable architecture solutions.
• Create architecture blueprints, reference architectures, technology roadmaps, governance standards and technical decision records.
• Lead architecture reviews, solution governance councils and technical design workshops.
• Provide technical leadership, mentoring, troubleshooting and architectural guidance to data engineering and platform teams.
• Evaluate emerging technologies and recommend architectural improvements aligned with business objectives.
• Support platform operationalization, reliability engineering, scalability planning and enterprise adoption strategies.

Skills

Databricks

AWS

PySpark

Spark SQL

Delta Lake

Unity Catalog

Cloud Architecture

Security & Governance

Terraform / IaC

Serverless Computing

Tools

Databricks Lakehouse

Delta Lake

Unity Catalog

Databricks SQL

Workflows

Terraform

AWS CloudFormation

Job description

About Position:

We are seeking an experienced AWS Databricks Architect with 13-20 years of experience in designing and implementing enterprise-scale cloud data platforms. The ideal candidate will possess deep expertise in Databricks Lakehouse architecture, Delta Lake, Unity Catalog, AWS data services, data modernization, governance, security, and large-scale migration programs. This role requires a strong architecture mindset combined with hands‑on technical leadership to deliver scalable, secure, cost‑optimized, and business‑aligned cloud data solutions.

• Role: AWS Databricks Architect


• Location: All Persistent Locations


• Experience: 13 to 20 Years


• Job Type: Full-Time Employment

What You'll Do:

• Design enterprise‑scale Lakehouse and Data Warehouse architectures using Databricks, Delta Lake, Unity Catalog, and Amazon S3.


• Define target‑state architecture covering Databricks workspaces, networking, storage, compute, governance, security, disaster recovery, and environment strategy.


• Architect secure integrations between Databricks and AWS services including Amazon S3, IAM, Glue, Redshift, Lambda, Kinesis, RDS, DMS, Secrets Manager, KMS, CloudWatch, and VPC.


• Design and oversee batch and real‑time data ingestion, transformation and processing pipelines using PySpark, Spark SQL, Databricks Workflows, Structured Streaming, and Delta Live Tables.


• Lead data platform migration and modernization initiatives from on‑premises and legacy environments to AWS Databricks platforms.


• Define and govern Medallion Architecture, data models, data quality frameworks, metadata standards, lineage and governance controls.


• Establish enterprise security standards including IAM roles, encryption, secrets management, least‑privilege access controls, private networking and auditability practices.


• Drive performance tuning and cost optimization across clusters, serverless compute, SQL warehouses, storage architectures and platform workloads.


• Define CI/CD frameworks, Infrastructure as Code standards, release processes, testing strategies, monitoring frameworks and operational support models.


• Collaborate with business stakeholders, product owners, infrastructure teams, security teams and engineering leaders to translate requirements into scalable architecture solutions.


• Create architecture blueprints, reference architectures, technology roadmaps, governance standards and technical decision records.


• Lead architecture reviews, solution governance councils and technical design workshops.


• Provide technical leadership, mentoring, troubleshooting and architectural guidance to data engineering and platform teams.


• Evaluate emerging technologies and recommend architectural improvements aligned with business objectives.


• Support platform operationalization, reliability engineering, scalability planning and enterprise adoption strategies.

Expertise You'll Bring:

• 13 to 20 years of overall IT experience, including significant experience in enterprise data architecture and solution architecture roles.


• Extensive hands‑on experience designing and implementing Databricks-based data platforms on AWS.


• Deep expertise in Databricks Lakehouse Platform, Delta Lake, Unity Catalog, Databricks SQL, Workflows, cluster policies and serverless computing.


• Strong AWS architecture experience covering Amazon S3, IAM, VPC, Glue, Redshift, Kinesis, Lambda, RDS, DMS, CloudWatch, KMS and Secrets Manager.


• Advanced proficiency in PySpark, Apache Spark, Spark SQL, Python, SQL and distributed data processing architectures.


• Strong experience designing enterprise data lakes, data warehouses, Lakehouse platforms, ETL/ELT frameworks and streaming data ecosystems.


• Proven expertise in leading cloud migration, data modernization and enterprise transformation initiatives.


• Deep understanding of cloud security, governance, compliance, networking, identity management, resilience and data privacy principles.


• Experience implementing Infrastructure as Code using Terraform, AWS CloudFormation, Git and automated deployment pipelines.


• Strong experience in performance tuning, scalability planning, workload optimization, reliability engineering and cloud cost management.


• Expertise in designing data governance frameworks, metadata management, data lineage, cataloging and data quality controls.


• Ability to communicate complex technical concepts effectively to business stakeholders, technical teams and executive leadership.


• Strong leadership experience in architecture governance, solution reviews, technical mentoring and enterprise platform strategy.


• Excellent stakeholder management, presentation, problem‑solving and decision‑making capabilities.


• Ability to balance architectural standards, business objectives, delivery timelines and operational realities.

Benefits:

• Competitive salary and benefits package


• Culture focused on talent development with quarterly growth opportunities and company-sponsored higher education and certifications


• Opportunity to work with cutting‑edge technologies


• Employee engagement initiatives such as project parties, flexible work hours and Long Service awards


• Annual health check‑ups


• Insurance coverage: group term life, personal accident, and Mediclaim hospitalization for self, spouse, two children, and parents

Values-Driven, People-Centric & Inclusive Work Environment:

Persistent is dedicated to fostering diversity and inclusion in the workplace. We invite applications from all qualified individuals, including those with disabilities, and regardless of gender or gender preference. We welcome diverse candidates from all backgrounds.

• We support hybrid work and flexible hours to fit diverse lifestyles.


• Our office is accessibility‑friendly, with ergonomic setups and assistive technologies to support employees with physical disabilities.


• If you are a person with disabilities and have specific requirements, please inform us during the application process or at any time during your employment

Let's unleash your full potential at Persistent - persistent.com/careers

"Persistent is an Equal Opportunity Employer and prohibits discrimination and harassment of any kind."\