Lead Data Engineer: Financial Data Ops & AI-Driven Pipelines

Happen Bank; San Francisco (CA); Hybrid; USD 190,000 - 220,000; Full time

Benefits offered by this job

Relocation assistance

Medical, dental and vision plans

401(k) match

Parental leave

Job summary

Happen Bank’s Data Operations Center seeks a Lead Data Engineer in San Francisco to own critical financial data pipelines and drive automation. You will bridge engineering, product, and business stakeholders while shaping data strategy and platform improvements.

You’ll lead onshore and offshore engineers, deploy Databricks, dbt, and Dagster, and push AI-enabled quality, monitoring, and efficiency across the team. Hybrid in-person collaboration required.

Qualifications

• 8+ years of data engineering experience with 2+ years in technical lead roles.
• Hands-on with AI tools to improve output and quality.
• Strong SQL, data modeling, and data warehouse concepts.
• Experience with orchestration tools (Airflow/Oozie/Dagster) and distributed processing (Spark).
• Knowledge of AWS data services and modern platforms (Snowflake/Databricks).
• Ability to translate business needs into scalable data solutions.
• Proven collaboration across finance, product, and engineering teams.

Responsibilities

• Lead a cross-functional scrum team of data engineers, setting priorities and removing blockers.
• Own architecture and maintenance of financial data pipelines supporting GL automation, investor reporting, and tax workflows.
• Partner with finance, accounting, and operations to translate requirements into scalable solutions.
• Drive adoption of modern data platform capabilities and maintain production systems.
• Identify opportunities to leverage AI for QA, code review, and performance optimization.
• Build monitoring, alerting, and observability to ensure pipeline reliability and rapid incident response.
• Establish coding standards and data quality frameworks at scale.
• Use AI to create runbooks, diagnostic agents, and self-service tools for L1 support.

Skills

Data engineering leadership

SQL & data modeling

Airflow / Dagster / Oozie

AWS (EMR, S3, Redshift)

Databricks / Spark

AI tooling for development

Cross-functional collaboration

Education

Bachelor's degree

Tools

Databricks

dbt

Dagster

Airflow

Spark / PySpark

Snowflake

Elementary

Job description

Happen Bank’s Data Operations Center seeks a Lead Data Engineer in San Francisco to own critical financial data pipelines and drive automation. You will bridge engineering, product, and business stakeholders while shaping data strategy and platform improvements.

You’ll lead onshore and offshore engineers, deploy Databricks, dbt, and Dagster, and push AI-enabled quality, monitoring, and efficiency across the team. Hybrid in-person collaboration required.