Databricks Architect

CAI; Indiana; Hybrid; USD 150,000 - 210,000; Full time

Job summary

CAI is seeking a Databricks Architect to lead design, implementation, governance, and optimization of enterprise-scale data and AI platforms on Databricks. This full-time, remote role requires hands-on expertise and strategic leadership across AWS and Azure.

You will define architecture standards, mentor teams, and collaborate with stakeholders to deliver scalable, secure Lakehouse solutions and modern data pipelines for analytics and AI workloads.

Qualifications

• Bachelor's degree or higher in Computer Science, IT, Engineering, Data Science, or a related field.
• 10+ years of experience in data engineering, data architecture, cloud architecture, or enterprise technology leadership roles.
• 5+ years of hands-on experience with Databricks and modern cloud-based data platforms.
• Extensive experience designing and implementing enterprise-scale data lake, data warehouse, and Lakehouse architectures.
• Strong expertise in Databricks, Delta Lake, Unity Catalog, Workflows, and Databricks SQL.
• Hands-on experience with AWS services such as S3, IAM, Glue, Lambda, EC2, RDS, Redshift, CloudWatch, and networking services.
• Experience designing data pipelines using Spark, PySpark, SQL, and modern orchestration frameworks.
• Deep understanding of data governance, security, metadata management, cataloging, lineage, and compliance requirements.
• Experience with CI/CD, Infrastructure as Code, DevOps, and platform automation practices.
• Strong knowledge of analytics, machine learning, and AI data platform requirements.
• Experience establishing enterprise architecture standards, patterns, and best practices.
• Proven ability to design highly available, scalable, and cost-efficient cloud architectures.
• Strong stakeholder management skills with experience engaging executive leadership, business stakeholders, and technical teams.

Responsibilities

• Define and own the enterprise Databricks architecture strategy, ensuring scalability, security, reliability, and alignment with business and AI objectives.
• Design modern data platforms leveraging Databricks Lakehouse, Delta Lake, Unity Catalog, and AWS services
• Lead the architecture and modernization of data warehouses, data lakes, and analytical platforms into scalable Lakehouse architectures
• Establish enterprise standards, frameworks, and best practices for data ingestion, transformation, orchestration, governance, and consumption
• Design and implement secure, high-performance batch and real-time data processing solutions
• Develop reference architectures for data engineering, advanced analytics, machine learning, and generative AI workloads
• Define data governance, lineage, metadata management, security, and compliance standards across the data platform
• Architect scalable and reusable data models that support reporting, analytics, operational intelligence, and AI use cases
• Guide implementation teams on platform setup, workspace design, networking, access controls, monitoring, and operational readiness
• Drive platform optimization initiatives related to compute utilization, workload management, storage performance, and cost efficiency
• Lead architectural reviews and provide oversight for critical data and AI initiatives
• Collaborate with business stakeholders, product owners, data engineers, ML engineers, and cloud teams to translate requirements into technical solutions
• Establish observability frameworks for monitoring platform health, performance, data quality, and usage patterns
• Evaluate emerging Databricks capabilities and recommend adoption strategies that enhance business value
• Develop migration strategies and execution roadmaps for legacy data platforms transitioning to Databricks
• Define disaster recovery, business continuity, backup, and resilience requirements for enterprise data platforms
• Drive architecture governance through design reviews, standards enforcement, and continuous improvement initiatives
• Mentor engineering teams and provide technical leadership on complex architecture decisions
• Deliver executive-level architecture documentation, roadmaps, solution blueprints, and technology recommendations
• Hands-on experience with both AWS and Azure cloud platforms, including architecture design, security, networking, storage, governance, and data services
• Strong understanding of multi-cloud data platform strategies and the ability to architect solutions across AWS and Azure environments
• Experience integrating Databricks with AWS and Azure native services to support enterprise analytics, data engineering, and AI use cases

Skills

Databricks

AWS

Azure

Delta Lake

Lakehouse

CI/CD

Leadership

Security

Governance

Multi-cloud

Education

Bachelor's degree or higher in Computer Science/IT/Engineering/Data Science

Tools

Databricks SQL

Unity Catalog

Workflows

S3

Glue

Redshift

CloudWatch

Job description

Databricks Architect

Req number:

R8699

Employment type:

Full time

Worksite flexibility:

Remote

Who we are

CAI is a global services firm with over 9,000 associates worldwide and a yearly revenue of $1.3 billion+. We have over 40 years of excellence in uniting talent and technology to power the possible for our clients, colleagues, and communities. As a privately held company, we have the freedom and focus to do what is right—whatever it takes. Our tailor-made solutions create lasting results across the public and commercial sectors, and we are trailblazers in bringing neurodiversity to the enterprise.

Job Summary

We are looking for a motivated Databricks Architect ready to take us to the next level! If you have lead the design, implementation, governance, and optimization of enterprise-scale data and AI platforms on Databricks and are looking for your next career move.

Job Description

We are looking for a Databricks Architect to lead the design, implementation, governance, and optimization of enterprise-scale data and AI platforms on Databricks. This position will be full-time and remote.

What You’ll Do

• Define and own the enterprise Databricks architecture strategy, ensuring scalability, security, reliability, and alignment with business and AI objectives
• Design modern data platforms leveraging Databricks Lakehouse, Delta Lake, Unity Catalog, and AWS services
• Lead the architecture and modernization of data warehouses, data lakes, and analytical platforms into scalable Lakehouse architectures
• Establish enterprise standards, frameworks, and best practices for data ingestion, transformation, orchestration, governance, and consumption
• Design and implement secure, high-performance batch and real-time data processing solutions
• Develop reference architectures for data engineering, advanced analytics, machine learning, and generative AI workloads
• Define data governance, lineage, metadata management, security, and compliance standards across the data platform
• Architect scalable and reusable data models that support reporting, analytics, operational intelligence, and AI use cases
• Guide implementation teams on platform setup, workspace design, networking, access controls, monitoring, and operational readiness
• Drive platform optimization initiatives related to compute utilization, workload management, storage performance, and cost efficiency
• Lead architectural reviews and provide oversight for critical data and AI initiatives
• Collaborate with business stakeholders, product owners, data engineers, ML engineers, and cloud teams to translate requirements into technical solutions
• Establish observability frameworks for monitoring platform health, performance, data quality, and usage patterns
• Evaluate emerging Databricks capabilities and recommend adoption strategies that enhance business value
• Develop migration strategies and execution roadmaps for legacy data platforms transitioning to Databricks
• Define disaster recovery, business continuity, backup, and resilience requirements for enterprise data platforms
• Drive architecture governance through design reviews, standards enforcement, and continuous improvement initiatives
• Mentor engineering teams and provide technical leadership on complex architecture decisions
• Deliver executive-level architecture documentation, roadmaps, solution blueprints, and technology recommendations
• Hands-on experience with both AWS and Azure cloud platforms, including architecture design, security, networking, storage, governance, and data services
• Strong understanding of multi-cloud data platform strategies and the ability to architect solutions across AWS and Azure environments
• Experience integrating Databricks with AWS and Azure native services to support enterprise analytics, data engineering, and AI use cases

What You’ll Need

Required:

• Bachelor's degree or higher in Computer Science, Information Technology, Engineering, Data Science, or a related field
• 10+ years of experience in data engineering, data architecture, cloud architecture, or enterprise technology leadership roles
• 5+ years of hands-on experience with Databricks and modern cloud-based data platforms
• Extensive experience designing and implementing enterprise-scale data lake, data warehouse, and Lakehouse architectures
• Strong expertise in Databricks, Delta Lake, Unity Catalog, Workflows, and Databricks SQL
• Hands-on experience with AWS services such as S3, IAM, Glue, Lambda, EC2, RDS, Redshift, CloudWatch, and networking services
• Experience designing data pipelines using Spark, PySpark, SQL, and modern orchestration frameworks
• Deep understanding of data governance, security, metadata management, cataloging, lineage, and compliance requirements
• Experience with CI/CD, Infrastructure as Code, DevOps, and platform automation practices
• Strong knowledge of analytics, machine learning, and AI data platform requirements
• Experience establishing enterprise architecture standards, patterns, and best practices
• Proven ability to design highly available, scalable, and cost-efficient cloud architectures
• Strong stakeholder management skills with experience engaging executive leadership, business stakeholders, and technical teams
• Ability to evaluate trade-offs and make architecture decisions in complex and ambiguous environments
• Strong communication and presentation skills with the ability to influence technical and non-technical audiences
• Proven leadership experience driving large-scale transformation and modernization initiatives

Preferred :

• Knowledge of Databricks Genie, AI/BI capabilities, and natural language-driven analytics solutions
• Databricks Certified Data Engineer Professional certification
• Databricks Certified Machine Learning Professional certification
• Databricks Certified Platform Architect certification
• AWS Solutions Architect Associate certification

Physical Demands

• Ability to safely and successfully perform the essential job functions
• Sedentary work that involves sitting or remaining stationary most of the time with occasional need to move around the office to attend meetings, etc.
• Ability to conduct repetitive tasks on a computer, utilizing a mouse, keyboard, and monitor

Reasonable accommodation statement

If you require a reasonable accommodation in completing this application, interviewing, completing any pre-employment testing, or otherwise participating in the employment selection process, please direct your inquiries to [e-mail] or [phone].