Senior Databricks Architect: Data Lakehouse & Pipelines

Remote; San Francisco (CA); On-site; USD 150,000 - 190,000; Full time

Job summary

Bounteous is seeking a Senior Databricks Engineer to design, build, and optimize data platforms on the Databricks Lakehouse. You will own production-grade data pipelines, partner with analytics teams, and set engineering standards for performance and cost efficiency across our cloud data estate.

You will architect medallion data architectures, optimize Spark workloads, manage Unity Catalog, and collaborate with data scientists to deliver reliable data products while mentoring junior engineers.

Qualifications

• 5+ years of data engineering experience with production solutions on Databricks and Spark.
• Proficiency in Python (PySpark) and advanced SQL.
• Deep experience with Delta Lake, Unity Catalog, and medallion architecture.
• Strong cloud experience and data services on AWS, Azure, or GCP.

Responsibilities

• Architect, build, and maintain scalable ETL/ELT pipelines on the Databricks Lakehouse Platform using PySpark and Delta Lake.
• Design medallion architecture, enforce data quality, governance, and lineage standards.
• Optimize Spark jobs and cluster configurations for performance and cost.
• Manage Unity Catalog for access control and cross-workspace sharing.
• Build workflows using Databricks Workflows, Delta Live Tables, and CI/CD pipelines.
• Collaborate with data scientists and analysts to translate requirements into data products.
• Mentor junior data engineers and conduct code reviews; monitor production pipelines.

Skills

Databricks

PySpark

Delta Lake

Unity Catalog

Spark SQL

Python

SQL

Cloud (AWS/Azure/GCP)

CI/CD

Git

ETL/ELT

Data modeling

Education

Bachelor's degree in Computer Science or Engineering

Tools

Airflow

dbt

Delta Live Tables

MLflow

Kubernetes

Job description

Bounteous is seeking a Senior Databricks Engineer to design, build, and optimize data platforms on the Databricks Lakehouse. You will own production-grade data pipelines, partner with analytics teams, and set engineering standards for performance and cost efficiency across our cloud data estate.

You will architect medallion data architectures, optimize Spark workloads, manage Unity Catalog, and collaborate with data scientists to deliver reliable data products while mentoring junior engineers.