Lead GCP Data Platform Architect (Real-Time)

IG Group; Greater London; Hybrid; GBP 120,000 - 170,000; Full time

Benefits offered by this job

Growth opportunities

Mentoring by leaders

Clear career progression

Diversity & inclusion programs

Volunteering time off

Job summary

IG Group is seeking a Principal Data Engineer to own the end-to-end architecture of our Google Cloud data platform, including BigQuery, Kafka ingestion, and batch/real-time processing. You will drive the migration from the legacy AWS/on-prem systems to GCP-native capabilities, defining platform contracts and self-service patterns for mature divisions to build trusted models using Core Lake data.

You will lead a team of data engineers, ensuring data quality, observability, and SLA/SLO compliance

Qualifications

• Hands-on experience designing and operating large-scale data pipelines on GCP.
• Experience leading or mentoring a team of data engineers.
• Strong data quality practices: contracts, observability, SLA/SLO definitions.
• SQL data transformations and dbt usage in production pipelines.
• Comfortable communicating with non-technical stakeholders in a regulated financial services environment.

Responsibilities

• Own the technical architecture of IG’s GCP data platform end-to-end, including Core Lake, batch, and real-time ingestion.
• Lead delivery of complex, high-risk work within the GCP consolidation program; migrate legacy AWS/on-premises to GCP-native capabilities.
• Define platform contracts, interfaces, and self-service patterns for mature divisions to reuse safely.
• Drive data quality with contracts, observability, monitoring, and automated remediation across layers.
• Set engineering standards through reviews, mentoring, and reusable frameworks across squads in multiple regions.
• Maintain delivery rigour: shape sprint cadences, monitor pipeline SLAs, and report metrics to Head of Data Engineering.

Skills

GCP Data Pipelines

Kafka

Team Mentoring

Data Quality

dbt

SQL Transformations

Cross-functional Communication

Regulated FinServ

Tools

BigQuery

Airflow

GCS

dbt

Job description

IG Group is seeking a Principal Data Engineer to own the end-to-end architecture of our Google Cloud data platform, including BigQuery, Kafka ingestion, and batch/real-time processing. You will drive the migration from the legacy AWS/on-prem systems to GCP-native capabilities, defining platform contracts and self-service patterns for mature divisions to build trusted models using Core Lake data.

You will lead a team of data engineers, ensuring data quality, observability, and SLA/SLO compliance