Director, Databricks Platform Architect - Unity Catalog Enablement

Infinitive Inc; McLean (VA); On-site; USD 170,000 - 215,000; Full time

Job summary

Infinitive is a data & AI consultancy helping global brands leverage insights, innovation, and efficiency. We seek an architect to define a unified, scalable Databricks-based data platform with Unity Catalog as the governance core, enabling secure data access across multiple domains.

You will shape enterprise standards, drive adoption, and mentor teams on best practices. The role emphasizes designing a self-service layer, implementing metadata, RBAC/ABAC, and data masking to meet regulatory

Qualifications

• Bachelor’s or Master’s degree in CS/Engineering or related field.
• 8+ years in data architecture or platform engineering.
• Deep hands-on Databricks Unity Catalog governance and metadata management.
• Proficiency in Python (PySpark) and SQL; cloud platforms (AWS/Azure/GCP).
• Strong API engineering with REST/GraphQL/OpenAPI and security/observability.
• Expert-level data governance knowledge and regulatory compliance.
• Excellent communication and influence across executive, technical, and risk audiences.

Responsibilities

• Define end-to-end architecture for a Databricks-based data platform at scale.
• Design a self-service platform layer using Unity Catalog for data discovery and access.
• Create patterns and references to drive enterprise reuse and standardization.
• Lead Unity Catalog governance: catalog hierarchies, access controls, and lineage.
• Implement metadata, RBAC/ABAC, and data masking for regulatory needs.
• Develop observability, FinOps, and cost-optimization for the platform.
• Architect REST/GraphQL APIs for governed data access and metadata integration.
• Guide Lakehouse-aligned data models and Spark/ETL/ELT pipelines with Unity Catalog.
• Collaborate with engineering, data science, and risk teams to align with business outcomes.
• Mentor engineers on Databricks and best practices; contribute to architecture groups.
• Promote easy adoption via design patterns and documentation.

Skills

Databricks, Spark & Delta Lake

Unity Catalog

Python (PySpark)

SQL

Cloud platforms (AWS/Azure/GCP)

API design REST/GraphQL/OpenAPI

Data governance

Communication

Education

Bachelor’s/Master’s in Computer Science or Engineering

Tools

Terraform/IaC

MLflow

Databricks Certified Data Engineer

AWS/Azure/GCP Architect certifications

Kafka / Structured Streaming

Job description

About Infinitive

Infinitive is a data & AI consultancy that enables global brands to deliver results through insights, innovation, and efficiency. We possess deep industry and technology expertise to drive and sustain adoption of new capabilities. We match our people and personalities to our clients' culture while bringing the right mix of talent and skills to enable high return on investment.

Infinitive has been named “Best Small Firms to Work For” by Consulting Magazine eight times, and has also been named a Washington Post Top Workplace, Washington Business Journal Best Places to Work, and Virginia Business Best Places to Work.

Role Overview

This architect will define and shape a unified platform service that enables scalable, governed, and cost-efficient data access across the bank. The ideal candidate will influence enterprise design standards and technical adoption by making Databricks Unity Catalog the effortless, observable, and default foundation for data integration, governance, and analytics across all business domains.

Key Responsibilities

Platform Vision & Architecture

• Define and champion the end-to-end architecture for the bank’s Databricks-based data platform, ensuring scalability, security, cost efficiency, and ease of adoption.
• Design a self-service platform layer that leverages Databricks Unity Catalog to deliver seamless data discovery, access, and observability across all environments.
• Establish architectural patterns and reference implementations that encourage enterprise-wide reuse and standardization.

Unity Catalog Strategy & Enablement

• Lead the design and implementation ofDatabricks Unity Catalog as the central governance plane—defining catalog hierarchies, fine-grained access controls, and cross-environment lineage.
• Evaluate and implement metadata, RBAC/ABAC, and data masking capabilities to meet regulatory and compliance requirements (e.g., GLBA, GDPR, HIPAA).
• Define the template architecture that allows Unity Catalog to operate as a scalable and cost-effective shared service across lines of business.

Scalability, Cost, and Observability

• Engineer platform capabilities that provide deep visibility into compute, storage, and catalog operations through integrated observability, monitoring, and FinOps practices.
• Develop resource optimization strategies to balance performance and cost while maintaining compliance and SLAs.
• Establish metrics, dashboards, and alerts to ensure the platform scales predictably under enterprise workloads.

API and Integration Design

• Architect streamlined RESTful/GraphQL APIs for secure, governed data access and metadata integration.
• Ensure interoperability with enterprise systems, APIs, and external data consumers using modern, consistent, and documented integration patterns.

Data Modeling & Pipeline Strategy

• Guide teams in buildingLakehouse-aligned data models that maximize reuse and governance.
• Oversee design of ETL/ELT architectures (Spark, PySpark, SQL) that integrate seamlessly with Unity Catalog for lineage and access tracking.

Collaboration & Influence

• Partner with engineering, data science, and risk teams to align platform design with business outcomes and regulatory expectations.
• Influence architecture steering committees and platform engineering groups to adopt the Databricks foundation as a managed, enterprise-wide service.
• Promote a culture of easy adoption through clear design patterns, documentation, and working sessions.

Technical Leadership & Mentorship

• Mentor engineers and architects on Databricks, Unity Catalog, and best practices for cost, scale, and observability.
• Contribute to internal architecture communities and upskill teams across multiple domains.

Required Skills & Qualifications

• Education:Bachelor’s or Master’s degree in Computer Science, Engineering, or related field.
• Experience:8+ years in data architecture or platform engineering, including experience designing enterprise-scale, distributed data environments.
• Databricks Expertise:Deep hands-on knowledge ofDatabricks, Delta Lake, Apache Spark, and Lakehouse principles.
• Unity Catalog Mastery:Demonstrated success architecting and operationalizingDatabricks Unity Catalogfor enterprise governance, metadata management, and access control.
• Programming & Data:Advanced proficiency inPython (PySpark)andSQL; experience with cloud data platforms (AWS, Azure, or GCP).
• API Engineering:Strong background inAPI architecture (REST, GraphQL, OpenAPI)and applying best-in-class security and observability.
• Governance Knowledge:Expert-level understanding ofdata governance frameworks, data quality management, and regulatory compliance.
• Soft Skills:Outstanding communication and influence skills, with ability to advocate for design principles across executive, technical, and risk audiences.

Preferred Qualifications

• Experience deploying Databricks and cloud infrastructure usingTerraform or IaC frameworks.
• Familiarity withMLflowand model governance integration.
• Relevant certifications (Databricks Certified Data Engineer,AWS/Azure/GCP Architect).
• Experience withreal-time data streamingtechnologies (Kafka, Structured Streaming).

Infinitive is required by law in some jurisdictions to include a reasonable estimate of the compensation range for this role. The determination of this range includes various factors not limited to skill set, level, experience, relevant training, and licensure and certifications. Compensation decisions are dependent on the facts and circumstances of each case. A reasonable estimate of the current range for this role in the U.S. is $170,000.00 - $215,000.00.

Infinitive is an Equal Opportunity Employer. All qualified applicants will receive consideration for employment without regard to race, color, religion, sex, sexual orientation, gender identity, national origin, disability, protected veteran status, or any other characteristic protected by applicable federal, state, or local law.