AWS Databricks Architect

Persistent; Pune District; On-site; INR 3,500,000 - 5,500,000; Full time

Benefits offered by this job

Competitive salary

Learning and development opportunities

Cutting-edge technologies

Flexible work hours

Annual health check-ups

Comprehensive insurance coverage

Job summary

Persistent is seeking an experienced AWS Databricks Architect to design enterprise-scale cloud data platforms in Pune and other Persistent locations. The role requires 13–20 years of IT experience and deep expertise in Databricks Lakehouse, Delta Lake, Unity Catalog, and AWS data services.

You will lead the architecture of Databricks workspaces, security, governance, and migration programs, guiding teams through modern data lakehouse implementations on AWS.

Qualifications

• 13–20 years of enterprise IT or data architecture experience.
• Deep expertise in Databricks Lakehouse, Delta Lake, Unity Catalog on AWS.
• Experience with large-scale migration, governance, and security.
• Proven leadership in architecture reviews and mentoring.

Responsibilities

• Design enterprise-scale Lakehouse and Data Warehouse architectures using Databricks, Delta Lake, Unity Catalog, and Amazon S3.
• Define target-state architecture covering Databricks workspaces, networking, storage, compute, governance, security, disaster recovery, and environment strategy.
• Architect secure integrations between Databricks and AWS services including S3, IAM, Glue, Redshift, Lambda, Kinesis, RDS, DMS, Secrets Manager, KMS, CloudWatch, and VPC.
• Lead data platform migration and modernization initiatives from on-premises and legacy environments to AWS Databricks platforms.
• Define and govern Medallion Architecture, data models, data quality frameworks, metadata standards, lineage, and governance controls.
• Establish enterprise security standards including IAM roles, encryption, secrets management, least-privilege access controls, private networking, and auditability practices.
• Drive performance tuning and cost optimization across clusters, serverless compute, SQL warehouses, storage architectures, and platform workloads.

Skills

Databricks Lakehouse

AWS data services

Security & Governance

PySpark

Data architecture

Tools

Databricks

AWS

Terraform

CloudFormation

IAM

Job description

About Position:

We are seeking an experienced AWS Databricks Architect with 13-20 years of experience in designing and implementing enterprise-scale cloud data platforms. The ideal candidate will possess deep expertise in Databricks Lakehouse architecture, Delta Lake, Unity Catalog, AWS data services, data modernization, governance, security, and large-scale migration programs. This role requires a strong architecture mindset combined with hands-on technical leadership to deliver scalable, secure, cost-optimized, and business-aligned cloud data solutions.

• Role: AWS Databricks Architect


• Location: All Persistent Locations


• Experience: 13 to 20 Years


• Job Type: Full-Time Employment

What You'll Do:

• Design enterprise-scale Lakehouse and Data Warehouse architectures using Databricks, Delta Lake, Unity Catalog, and Amazon S3.


• Define target-state architecture covering Databricks workspaces, networking, storage, compute, governance, security, disaster recovery, and environment strategy.


• Architect secure integrations between Databricks and AWS services including Amazon S3, IAM, Glue, Redshift, Lambda, Kinesis, RDS, DMS, Secrets Manager, KMS, CloudWatch, and VPC.


• Design and oversee batch and real-time data ingestion, transformation, and processing pipelines using PySpark, Spark SQL, Databricks Workflows, Structured Streaming, and Delta Live Tables.


• Lead data platform migration and modernization initiatives from on-premises and legacy environments to AWS Databricks platforms.


• Define and govern Medallion Architecture, data models, data quality frameworks, metadata standards, lineage, and governance controls.


• Establish enterprise security standards including IAM roles, encryption, secrets management, least-privilege access controls, private networking, and auditability practices.


• Drive performance tuning and cost optimization across clusters, serverless compute, SQL warehouses, storage architectures, and platform workloads.


• Define CI/CD frameworks, Infrastructure as Code standards, release processes, testing strategies, monitoring frameworks, and operational support models.


• Collaborate with business stakeholders, product owners, infrastructure teams, security teams, and engineering leaders to translate requirements into scalable architecture solutions.


• Create architecture blueprints, reference architectures, technology roadmaps, governance standards, and technical decision records.


• Lead architecture reviews, solution governance councils, and technical design workshops.


• Provide technical leadership, mentoring, troubleshooting, and architectural guidance to data engineering and platform teams.


• Evaluate emerging technologies and recommend architectural improvements aligned with business objectives.


• Support platform operationalization, reliability engineering, scalability planning, and enterprise adoption strategies.

Expertise You'll Bring:

• 13 to 20 years of overall IT experience, including significant experience in enterprise data architecture and solution architecture roles.


• Extensive hands-on experience designing and implementing Databricks-based data platforms on AWS.


• Deep expertise in Databricks Lakehouse Platform, Delta Lake, Unity Catalog, Databricks SQL, Workflows, cluster policies, and serverless computing.


• Strong AWS architecture experience covering Amazon S3, IAM, VPC, Glue, Redshift, Kinesis, Lambda, RDS, DMS, CloudWatch, KMS, and Secrets Manager.


• Advanced proficiency in PySpark, Apache Spark, Spark SQL, Python, SQL, and distributed data processing architectures.


• Strong experience designing enterprise data lakes, data warehouses, Lakehouse platforms, ETL/ELT frameworks, and streaming data ecosystems.


• Proven expertise in leading cloud migration, data modernization, and enterprise transformation initiatives.


• Deep understanding of cloud security, governance, compliance, networking, identity management, resilience, and data privacy principles.


• Experience implementing Infrastructure as Code using Terraform, AWS CloudFormation, Git, and automated deployment pipelines.


• Strong experience in performance tuning, scalability planning, workload optimization, reliability engineering, and cloud cost management.


• Expertise in designing data governance frameworks, metadata management, data lineage, cataloging, and data quality controls.


• Ability to communicate complex technical concepts effectively to business stakeholders, technical teams, and executive leadership.


• Strong leadership experience in architecture governance, solution reviews, technical mentoring, and enterprise platform strategy.


• Excellent stakeholder management, presentation, problem-solving, and decision-making capabilities.


• Ability to balance architectural standards, business objectives, delivery timelines, and operational realities.

Benefits:

• Competitive salary and benefits package


• Culture focused on talent development with quarterly growth opportunities and company-sponsored higher education and certifications


• Opportunity to work with cutting-edge technologies


• Employee engagement initiatives such as project parties, flexible work hours, and Long Service awards


• Annual health check-ups


• Insurance coverage: group term life, personal accident, and Mediclaim hospitalization for self, spouse, two children, and parents

Values-Driven, People-Centric & Inclusive Work Environment:

Persistent is dedicated to fostering diversity and inclusion in the workplace. We invite applications from all qualified individuals, including those with disabilities, and regardless of gender or gender preference. We welcome diverse candidates from all backgrounds.

• We support hybrid work and flexible hours to fit diverse lifestyles.


• Our office is accessibility-friendly, with ergonomic setups and assistive technologies to support employees with physical disabilities.


• If you are a person with disabilities and have specific requirements, please inform us during the application process or at any time during your employment

Persistent is an Equal Opportunity Employer and prohibits discrimination and harassment of any kind.