Senior Data Platform Lead — Real-Time Finance

Airwallex-; San Francisco (CA); On-site; USD 180,000 - 300,000; Full time

Job summary

Airwallex in San Francisco is seeking a Senior Manager of Data Platform Engineering to lead the data backbone of our financial systems. You will set the direction for high-volume data ingestion, the Databricks lakehouse, and data orchestration to ensure real-time, reliable analytics.

You’ll build a high-performing team, establish platform standards, and partner with product, accounting, and infrastructure to deliver scalable, auditable data foundations.

Qualifications

• 8+ years in data platform, backend, or infrastructure engineering with leadership experience.
• Proven ability to grow senior engineers and technical leads.
• Deep expertise in data platforms, distributed systems, data modeling, and production data pipelines.
• Strong Python and SQL skills; hands-on with Spark, Databricks, Delta Lake, and dbt.
• Experience building large-scale ingestion systems with backfills, incremental processing, checkpointing, and recovery.
• Strong understanding of streaming architectures, including Kafka, CDC, and micro-batching.
• Experience with Dagster for data orchestration and lakehouse patterns.
• Operations experience on GCP; familiar with Terraform, Kubernetes/GKE, Helm, CI/CD.

Responsibilities

• Lead and grow a high-performing data platform engineering team.
• Set technical strategy for high-volume data ingestion and lakehouse architecture.
• Oversee evolution of Databricks lakehouse with Spark, Delta Lake, Unity Catalog, dbt, Python, and SQL.
• Define standards for data contracts, schema evolution, and data quality.
• Advance batch-to-real-time processing using Kafka, CDC, and Dagster orchestration.
• Own canonical financial models powering the accounting engine.
• Partner with infra and platform teams for observability and incident response.
• Guide infrastructure strategy across GCP, Terraform, Kubernetes/GKE, Helm, GitOps, CI/CD.
• Lead migrations from legacy pipelines to the lakehouse with parity validation.
• Drive performance and cost optimization across PostgreSQL and large data workloads.
• Collaborate with product, accounting, ML, and platform leadership.
• Set architecture direction via reviews, docs, and long-term roadmaps.
• Champion AI-assisted engineering practices to boost velocity while ensuring quality.

Skills

Leadership of engineering teams

Data platform architecture

Python programming

SQL proficiency

Distributed systems

Observability & SRE

Data modeling

GCP & Terraform

Tools

Databricks

Spark

Delta Lake

dbt

Kafka

Dagster

Kubernetes

GKE

Terraform

PostgreSQL

Python

Job description

Airwallex in San Francisco is seeking a Senior Manager of Data Platform Engineering to lead the data backbone of our financial systems. You will set the direction for high-volume data ingestion, the Databricks lakehouse, and data orchestration to ensure real-time, reliable analytics.

You’ll build a high-performing team, establish platform standards, and partner with product, accounting, and infrastructure to deliver scalable, auditable data foundations.