Senior Data Platform Architect - Databricks & Airflow

Centraprise; Irvine (CA); On-site; USD 180,000 - 240,000; Full time

Job summary

Centraprise in Irvine, CA is seeking a Senior Data Engineering Architect to design, scale, and optimize the enterprise data platform. You will drive the blueprints for the data estate using Databricks, dbt, and Airflow, bridging business strategy with technical implementation across pipelines and data products.

You will mentor engineers, enforce data quality, and collaborate with product and analytics teams to deliver scalable, cost‑effective solutions worldwide.

Qualifications

• 15+ years of experience in data engineering, data warehousing, and distributed systems.
• 10+ years in data engineering roles and related architecture responsibilities.
• 4+ years architecting production environments in the modern data stack.
• Strong Python (PySpark) and SQL programming capabilities.
• Deep expertise with Databricks Photon, Unity Catalog, Delta Lake, and DLT.
• Advanced dbt proficiency including macros, materializations, and multi‑project setups.
• Airflow scheduling, operators, dynamic task mapping, and scalable infra.
• Cloud experience with AWS, Azure, or GCP, plus IaC tooling like Terraform.

Responsibilities

• Design end‑to‑end lakehouse architectures on Databricks with Delta Lake and Unity Catalog.
• Establish governance, schema evolution, and fine‑grained data security patterns.
• Define standard data modeling frameworks (Kimball, Data Vault 2.0).
• Optimize infrastructure for price‑to‑performance across batch and streaming workloads.
• Build modular transformation frameworks using dbt Core/Cloud with Databricks.
• Standardize processing with PySpark, Delta Live Tables, and Spark SQL.
• Construct observable orchestration workflows with Apache Airflow.
• Model cross‑DAG dependencies, providers, and robust error handling.
• Drive DataOps maturity with CI/CD pipelines via GitHub Actions, GitLab CI, or Azure DevOps.
• Apply IaC patterns with Terraform and Databricks Asset Bundles (DABs).
• Embed automated data quality testing into dbt and Airflow lifecycles.
• Define SLIs/SLOs for uptime and data freshness; act as data engineering authority.
• Mentor senior and mid‑level engineers through code reviews and workshops.
• Collaborate with product managers, data scientists, and business leaders on data gaps.

Job description

Centraprise in Irvine, CA is seeking a Senior Data Engineering Architect to design, scale, and optimize the enterprise data platform. You will drive the blueprints for the data estate using Databricks, dbt, and Airflow, bridging business strategy with technical implementation across pipelines and data products.

You will mentor engineers, enforce data quality, and collaborate with product and analytics teams to deliver scalable, cost‑effective solutions worldwide.