Senior Databricks Data Platform Architect

Unison Group; San Francisco (CA); On-site; USD 180,000 - 240,000; Full time

Job summary

Unison Group, based in San Francisco, seeks a Senior Solution Architect with deep Databricks expertise to lead the design, architecture, and implementation of scalable cloud data platforms. You will translate customer requirements into practical data solutions and mentor engineering teams on data governance and platform optimization.

Responsibilities include designing Data Lakehouse architectures, developing Spark-based ETL/ELT pipelines, and guiding best practices for performance, security, and

Qualifications

• Deep hands-on experience designing and implementing enterprise-grade data platforms.
• Proven ability to translate business requirements into scalable Databricks architectures.
• Strong emphasis on data governance, security, and cost-efficient operations.

Responsibilities

• Collaborate with customers and stakeholders to gather requirements and design scalable data platform architectures.
• Design and implement modern Data Lakehouse solutions using Databricks.
• Develop and optimize ingestion, transformation, and processing pipelines with Apache Spark.
• Define data architecture following Medallion Architecture (Bronze, Silver, Gold).
• Lead design discussions, workshops, and architecture reviews; mentor teams on best practices.
• Optimize Spark workloads for performance, scale, and cost efficiency.
• Ensure data security, governance, and monitoring across platforms.
• Work with cross-functional teams to deliver end-to-end solutions.
• Support pre-sales with technical consulting and architecture diagrams.

Skills

Databricks

Python

SQL

Apache Spark

Data Lake

Lakehouse

Medallion Architecture

ETL/ELT Pipelines

Data Modeling

Partitioning

Optimization

Performance Tuning

Data Quality & Governance

CI/CD

Git

Agile

Logging & Monitoring

Security Best Practices

Tools

Delta Lake

Unity Catalog

Delta Live Tables

Databricks Workflows

MLflow

Job description

Unison Group, based in San Francisco, seeks a Senior Solution Architect with deep Databricks expertise to lead the design, architecture, and implementation of scalable cloud data platforms. You will translate customer requirements into practical data solutions and mentor engineering teams on data governance and platform optimization.

Responsibilities include designing Data Lakehouse architectures, developing Spark-based ETL/ELT pipelines, and guiding best practices for performance, security, and