Remote Databricks Architect - Lakehouse & AI Platform Lead

CAI Computer Aid, Inc.; United States; À distance; USD 180 000 - 260 000; Plein temps

Résumé du poste

CAI Computer Aid, Inc. is seeking an experienced Databricks Architect to lead the design, governance, and optimization of enterprise-scale data and AI platforms on Databricks.

The role is full-time and remote, offering collaboration with cloud teams and business stakeholders. You will define architecture strategy, implement modern Lakehouse architectures, and establish standards for data ingestion, processing, and security across AWS and Azure environments.

Qualifications

• Bachelor's degree or higher in a related field
• 10+ years in data engineering/architecture or leadership roles
• 5+ years with Databricks and modern cloud data platforms
• Experience designing enterprise-scale data lake, data warehouse, Lakehouse
• Strong skills with Databricks, Delta Lake, Unity Catalog, Workflows, Databricks SQL
• Hands-on with AWS services (S3, IAM, Glue, Lambda, EC2, RDS, Redshift)
• Data pipelines using Spark, PySpark, SQL, orchestration frameworks
• Knowledge of governance, security, metadata, cataloging, lineage, compliance
• Experience with CI/CD, IaC, DevOps, platform automation
• Leadership in enterprise architecture standards and platforms

Responsabilités

• Define and own enterprise Databricks architecture strategy for scalable, secure platforms
• Design data platforms using Databricks Lakehouse, Delta Lake, Unity Catalog, AWS services
• Lead modernization of data warehouses, data lakes, and analytics into Lakehouse architectures
• Establish standards and best practices for ingestion, transformation, governance, and consumption
• Design secure, high-performance batch and real-time data processing
• Develop reference architectures for data engineering, analytics, ML, and generative AI
• Define governance, metadata, security, and compliance standards across the platform
• Architect reusable data models supporting reporting, analytics and AI use cases
• Guide teams on platform setup, workspace design, access controls, monitoring
• Drive optimization for compute, storage, and cost; oversee critical initiatives
• Collaborate with stakeholders to translate requirements into technical solutions
• Establish observability frameworks for health, performance, data quality
• Evaluate Databricks capabilities; develop adoption strategies
• Develop migration roadmaps for legacy platforms to Databricks
• Define DR, backup, and resilience requirements
• Mentor engineering teams and deliver architecture docs and roadmaps
• Hands-on with AWS and Azure cloud platforms and security
• Understand multi-cloud data platform strategies across AWS/Azure
• Integrate Databricks with native services for enterprise analytics

Connaissances

Databricks

Cloud architecture

Data governance

Leadership

Stakeholder management

Formation

Bachelor's degree or higher

Outils

AWS

Azure

Unity Catalog

Databricks SQL

Description du poste

CAI Computer Aid, Inc. is seeking an experienced Databricks Architect to lead the design, governance, and optimization of enterprise-scale data and AI platforms on Databricks.

The role is full-time and remote, offering collaboration with cloud teams and business stakeholders. You will define architecture strategy, implement modern Lakehouse architectures, and establish standards for data ingestion, processing, and security across AWS and Azure environments.