Senior Data Platform Architect, GCP — Real-Time Pipelines

LTM; Sydney; On-site; AUD 150,000 - 210,000; Full time

Job summary

LTM in Sydney, Australia, seeks a senior data engineering lead to architect and deliver a Transactions Ingestion Data Platform on GCP, covering real-time and batch pipelines. You will work with the customer’s data teams to shape schemas for Spanner and BigQuery and drive MVP delivery.

Hands-on with Python/Java, CI/CD, Terraform, Cloud Composer, and data governance tools; experience with generative AI integrations in large enterprise environments is valued.

Qualifications

• In-depth knowledge of GCP data and analytics technologies.
• Experience designing large-scale transaction data platform architectures on GCP.
• Define solution architecture and detailed design, and perform hands-on implementation for data pipelines.

Responsibilities

• Lead technical architecture and solution design for a large FSI customer Transactions Ingestion Data Platform (Real Time and Batch).
• Interact with customer’s data engineering team to provide guidance on platform architecture, schemas (Spanner/BigQuery) and solutions.
• Develop and unit test ingestion/processing pipelines (e.g., Kafka to Spanner/BigQuery via Dataflow, GCS to Spanner/BigQuery) and handover to customer team.
• Perform pair programming with customer’s data engineers.
• Project manage sprint-based engagements, leading planning, backlog management, scope, risks, and MVP delivery.
• Collaborate with cross-disciplinary project team as tech lead for workstreams or entire project.

Skills

GCP data analytics

Data architecture

Data pipelines

Python

Java

CI/CD

Cloud Composer

Dataplex/Data Catalog

Tools

Terraform

Dataform

dbt

Kafka

Google Dataflow

Spanner

BigQuery

Job description

LTM in Sydney, Australia, seeks a senior data engineering lead to architect and deliver a Transactions Ingestion Data Platform on GCP, covering real-time and batch pipelines. You will work with the customer’s data teams to shape schemas for Spanner and BigQuery and drive MVP delivery.

Hands-on with Python/Java, CI/CD, Terraform, Cloud Composer, and data governance tools; experience with generative AI integrations in large enterprise environments is valued.