Principal Data Platform Engineer (Databricks)

Tally; San Francisco (CA); Hybrid; USD 197,000 - 230,000; Full time

Job summary

Happen Bank in San Francisco is seeking a Principal Data Engineer to own the technical architecture of our modern data and AI platform on Databricks. You will shape ingestion, data protection, and IaC standards to empower rapid, safe production pipelines across teams.

In this hybrid role, you’ll mentor senior engineers, drive platform reliability, and partner with data governance, security, and operations to deliver durable data capabilities at scale.

Qualifications

• 10+ years in software or data platform engineering.
• Bachelor's degree or higher.
• Experience with Terraform and CI/CD for data platforms.
• Ingestion capabilities across diverse sources including CDC and schema evolution.
• Proficient in Python, SQL, and JVM languages with Spark UDFs.
• Databricks expertise (Delta Lake, Unity Catalog, Auto Loader).
• Security and privacy engineering for sensitive data.
• Designing large-scale data systems processing billions of rows.
• Strong communication and stakeholder influence.
• AI tools in engineering workflows.

Responsibilities

• Own ingestion architecture and define a declarative, configuration-driven framework.
• Build reusable platform libraries with policy-driven data protection.
• Set infrastructure-as-code standards for safe deployment and lifecycle.
• Enhance reliability with monitoring, alerts, and data validation.
• Lead architectural decisions and engage vendors for gaps.
• Coordinate cross-functional delivery with governance and security.
• Mentor engineers and promote testing, docs, and release practices.

Skills

Data platform engineering

Python

SQL

Scala/Java

Databricks

Terraform

CI/CD

Security & privacy

Observability

Leadership

Education

Bachelor's degree or higher

Tools

Databricks

Delta Lake

Unity Catalog

Auto Loader

Job description

Happen Bank in San Francisco is seeking a Principal Data Engineer to own the technical architecture of our modern data and AI platform on Databricks. You will shape ingestion, data protection, and IaC standards to empower rapid, safe production pipelines across teams.

In this hybrid role, you’ll mentor senior engineers, drive platform reliability, and partner with data governance, security, and operations to deliver durable data capabilities at scale.