Databricks Platform Architect — Lakehouse & CI/CD Lead

Cooley LLP; New York (NY); On-site; USD 175,000 - 250,000; Full time

Benefits offered by this job

Medical benefits

Dental & Vision benefits

PTO 21 days

Paid holidays

Job summary

Cooley AI is seeking a Platform Architect to lead architecture for the Northstar Lakehouse on Databricks, reporting to the Platform & Operations function. This role focuses on designing, securing, and scaling the Databricks platform, including Unity Catalog, compute tier strategy, and CI/CD automation.

Collaboration with Data Engineering and governance teams is essential to align with the firm's AI ambitions.

Qualifications

• 4+ years directly applicable data platform engineering or architecture
• Deep Databricks expertise including Unity Catalog architecture and governance configuration
• Strong CI/CD experience including GitHub Actions, Databricks Asset Bundles, IaC patterns
• Experience with AWS including IAM, VPC, S3 and integration with Databricks workspaces
• Strong Python and SQL proficiency for platform automation and pipelines
• Bachelor’s degree and related qualifications are preferred

Responsibilities

• Own the Databricks platform architecture for the Northstar Lakehouse, including catalog and schema design, compute tier, and governance
• Design and maintain Unity Catalog governance architecture with access controls and audit logging
• Define and maintain compute architecture standards across the platform
• Evaluate new Databricks capabilities and advise on adoption timing
• Maintain platform observability and CI/CD framework for Databricks pipelines
• Collaborate with DataOps and Data Engineering to scale platform usage
• Document architectural decisions and maintain living platform documentation

Skills

Databricks platform architecture

Unity Catalog governance

CI/CD

Python

SQL

AWS cloud fundamentals

Platform architecture documentation

Security and access control design

Education

Bachelor’s degree

Tools

GitHub Actions

Databricks Asset Bundles

Terraform

AWS CloudFormation

Delta Lake

Job description

Cooley AI is seeking a Platform Architect to lead architecture for the Northstar Lakehouse on Databricks, reporting to the Platform & Operations function. This role focuses on designing, securing, and scaling the Databricks platform, including Unity Catalog, compute tier strategy, and CI/CD automation.

Collaboration with Data Engineering and governance teams is essential to align with the firm's AI ambitions.