Databricks Architect

Jade Business Services (JBS); Houston (TX); Hybride; USD 180 000 - 260 000; Plein temps

Résumé du poste

Jade Business Services (JBS) seeks a Senior Databricks Architect / Data & AI Architect to lead enterprise-scale Data & AI transformations from Houston. You will define end-to-end Lakehouse architectures, set technical strategy, and engage with client CTOs and VPs.

The ideal candidate has 10+ years in data platforms, deep Databricks expertise, and hands-on experience across Azure and GCP, BigQuery, and governance tools such as Unity Catalog. Hybrid work is offered.

Qualifications

• 10+ years in Data Engineering, Data Platform Architecture, Cloud Architecture, or Enterprise Solution Architecture.
• Databricks expertise with 5+ years in architecture.
• Presales, RFP/RFI authoring, and executive-level presentations.
• Bachelor’s or Master’s degree in Computer Science, Information Systems, Software Engineering, or related field.

Responsabilités

• Architect, design, and validate enterprise-grade Databricks Lakehouse and AI solutions.
• Design robust data pipelines using Lakeflow tools.
• Establish unified governance, RBAC/ABAC, and data lineage across catalogs.
• Lead architecture for RFP/RFI responses and client engagements with CTOs/VPs.

Connaissances

Strategic thinking

Executive communication

Presales experience

Architectural leadership

Formation

Bachelor’s or Master’s in CS/IS/SE

Outils

Databricks Lakeflow

Delta Lake

Delta UniForm

Apache Iceberg

Unity Catalog

Databricks Vector Search

Terraform

MLflow

Azure Synapse

GCP BigQuery

Description du poste

Senior Databricks Architect / Data & AI Architect

Location: Houston, TX (Hybrid)

Employment Type: Full-Time

Experience Level: Senior / Principal (10+ Years)

Role Summary

Jade Business Services (JBS) is seeking an experienced, visionary Senior Databricks Architect / Data & AI Architect to spearhead enterprise-scale Data and AI transformations. In this role, you will define end‑to‑end Lakehouse architectures, set technical strategy, drive presales/RFP technical responses, and engage directly with client executive leadership (VP/C‑suite).

The ideal candidate blends deep, hands‑on architectural expertise across the modern Databricks ecosystem—including real‑time streaming, Lakeflow pipelines, Unity Catalog governance, and agentic AI architectures— with proven experience in cloud platforms (Azure, GCP), data warehousing (BigQuery), and strategic consulting.

Key Responsibilities

• Architect, design, and validate enterprise‑grade Databricks Lakehouse, Real‑Time Streaming, Data Engineering, and Generative/Agentic AI solutions.
• Design robust data pipelines leveraging Lakeflow Designer, Lakeflow Connect, and Lakeflow Declarative Pipelines.
• Establish unified enterprise governance, access controls, auditability, and data lineage across catalogs using Unity Catalog (RBAC/ABAC).
• Evaluate architectural trade‑offs across performance, scalability, security posture, and cloud compute/storage costs (FinOps).
• Define interoperability strategies across Delta Lake and Apache Iceberg open table formats.

Presales, Consulting & Technical Leadership

• Lead the architecture stream for major RFP, RFI, and client proposal responses, delivering high‑impact technical narratives and architectural diagrams.
• Serve as the trusted technical advisor in high‑stakes discussions with client executives (CTOs, CDOs, VPs).
• Translate complex business problems into modular, cloud‑agnostic, and future‑proof Data & AI blueprints.
• Conduct architecture reviews, gap analyses, and proof‑of‑concept (PoC) implementations for strategic enterprise clients.

Required Technical Competencies

Core Databricks & AI Platform

• Pipeline Orchestration & Ingestion: In‑depth hands‑on experience with Databricks Lakeflow, Lakeflow Declarative Pipelines, Lakeflow Designer, and Lakeflow Connect.
• Storage & Interoperability: Delta Lake, Delta UniForm, and Apache Iceberg interoperability.
• Unified Governance: Advanced Unity Catalog implementation, fine‑grained access control (RBAC/ABAC), dynamic masking, row‑level filtering, and system tables/lineage.
• AI & Agentic Systems: Databricks Vector Search / AI Search, Unity AI Gateway, Databricks Genie / Genie Code, Managed Agent Memory, and Agent Bricks.
• Core Compute & Frameworks: PySpark, Apache Spark Structured Streaming, MLflow, Retrieval‑Augmented Generation (RAG) frameworks, and agentic AI orchestration.
• Cloud Platforms: Strong architectural experience across Microsoft Azure (ADLS Gen2, Azure Synapse, AKS) and Google Cloud Platform (GCP) (GCS, Dataproc, BigQuery).
• Cross‑Platform Analytics: Hands‑on experience integrating Databricks with Google BigQuery, cross‑cloud querying, and data virtualization.
• DevOps & CI/CD: Terraform for Databricks/Cloud provisioning, Databricks Asset Bundles (DABs), and automated CI/CD pipeline deployments.

Qualifications & Experience

• Experience: 10+ years in Data Engineering, Data Platform Architecture, Cloud Architecture, or Enterprise Solution Architecture.
• Databricks Expertise: 5+ years of specialized Databricks architecture experience, demonstrating mastery of the latest platform features and architecture patterns.
• Presales & Advisory: Proven track record in RFP/RFI authoring, presales solution design, client‑facing consulting, and executive‑level presentations.
• Education: Bachelor’s or Master’s degree in Computer Science, Information Systems, Software Engineering, or related field (or equivalent professional experience).

Certifications (Preferred)

• Databricks Certified Solutions Architect (Professional)
• Microsoft Certified: Azure Solutions Architect Expert / GCP Professional Data Engineer