Azure Databricks Lakehouse Architect | Data Platform Lead

CACI International; United States; Remote; USD 115,000 - 252,000; Full time

Job summary

CACI International is seeking a Lakehouse Architect / Databricks SME to support the DHS Office of Inspector General. You will design and implement a secure, scalable data lakehouse on Azure Databricks and Azure Data Lake Storage that serves as the authoritative data foundation for AI, analytics, and investigative workflows in a government cloud.

You will lead tiered storage, catalog integration, compute configurations, and governance while collaborating with data engineers, government

Qualifications

• Bachelor's degree plus 15 years of experience in data architecture/engineering or related field (equivalencies allowed).
• Ability to obtain Active DHS/EOD clearance as required.
• Deep expertise in Azure Databricks, Delta Lake, Unity Catalog, and medallion architectures.

Responsibilities

• Lead the design and implementation of a secure, scalable enterprise data lakehouse on Azure Databricks and Azure Data Lake Storage.
• Architect tiered storage zones (raw/bronze, curated/silver, analytics-ready/gold) with governance at each layer.
• Configure environments including workspace architecture, compute clusters, autoscaling, and cost optimization.
• Design data catalog integration, metadata management, and SQL-accessible tables/views for governed discovery and lineage.
• Support analytical models including SQL analytics, notebooks (Python/R/Scala), ML, and visualization integration.
• Migrate analyst workflows from local desktop processing to cloud-based notebooks and SQL analytics.
• Provide surge support for complex data architecture challenges and performance optimization.
• Develop ML/AI infrastructure including MLflow, feature stores, and integration between Databricks and AI services.
• Create training content and user enablement materials for investigators, analysts, and data scientists.
• Collaborate with government stakeholders, data engineers, and governance specialists to align with mission analytics requirements.

Skills

Analytical thinking

Stakeholder communication

Education

Bachelor's degree

Tools

Azure Databricks

Delta Lake

Unity Catalog

Azure Data Lake Storage

PySpark

SQL

MLflow

Job description

CACI International is seeking a Lakehouse Architect / Databricks SME to support the DHS Office of Inspector General. You will design and implement a secure, scalable data lakehouse on Azure Databricks and Azure Data Lake Storage that serves as the authoritative data foundation for AI, analytics, and investigative workflows in a government cloud.

You will lead tiered storage, catalog integration, compute configurations, and governance while collaborating with data engineers, government