Lead Databricks Data Engineer – Hands-On & Governance

U.S. Bank; Cupertino (CA); Sur place; USD 133 000 - 157 000; Plein temps

Avantages offerts par ce poste

Healthcare (medical, dental, vision)

401(k) employer-funded retirement plan

Paid vacation

Résumé du poste

U.S. Bank is seeking a technical leader to design and deliver scalable data products on a Databricks Lakehouse across Azure and AWS. This hands-on role will guide engineering, product, architecture, and governance teams and deliver trusted data for analytics, reporting, and AI use cases.

You will stay hands-on in engineering and development, shaping batch, streaming, and API pipelines, and defining standards for CI/CD, quality, and security. Collaboration across distributed teams is essential.

Qualifications

• Bachelor’s degree, or equivalent work experience.
• Six to eight years of relevant experience.

Responsabilités

• Lead and guide Databricks data engineering across Bronze, Silver, and Gold layers, with particular focus on trusted, reusable Silver and Gold data products.
• Translate business requirements into scalable data architecture, engineering designs, delivery standards, and technical acceptance criteria.
• Remain hands-on in engineering and development, including coding, code review, and code management, while guiding batch, CDC, streaming, API, and file-based data pipelines.
• Review data models, solution designs, and critical code for quality, security, scalability, performance, and alignment with engineering standards.
• Drive cross-cloud data integration and sharing across Azure, AWS, Databricks, and Snowflake.
• Embed data quality, lineage, governance, PII and PCI controls, monitoring, and reliability into data solutions.
• Establish engineering best practices for CI/CD, testing, source control, deployment, and observability.
• Provide technical leadership through influence across distributed onshore, offshore, and GCC engineering teams, establishing clear standards and guiding delivery without direct people-management responsibility.
• Partner with product, architecture, governance, platform, and business stakeholders to resolve technical dependencies, risks, and delivery challenges.

Connaissances

Python

PySpark

SQL

Spark SQL

ETL/ELT

Databricks

Delta Lake

Unity Catalog

Data pipelines

CI/CD

Git

Code review

Batch/Streaming

Azure

AWS

Power BI/DAX

Data governance

Data quality

PII/PCI controls

Cross-cloud data sharing

Formation

Bachelor’s degree, or equivalent work experience

Outils

Databricks

Delta Lake

Unity Catalog

Delta architecture

Databricks Workflows/Lakeflow

Terraform

Apache Airflow

Datadog

Tableau

Databricks Metric Views

Description du poste

U.S. Bank is seeking a technical leader to design and deliver scalable data products on a Databricks Lakehouse across Azure and AWS. This hands-on role will guide engineering, product, architecture, and governance teams and deliver trusted data for analytics, reporting, and AI use cases.

You will stay hands-on in engineering and development, shaping batch, streaming, and API pipelines, and defining standards for CI/CD, quality, and security. Collaboration across distributed teams is essential.