Data Engineer

InterGame Ltd; United Kingdom; Hybrid; GBP 65,000 - 100,000; Full time

Job summary

InterGame Ltd. is seeking a Senior Data Engineer to own the lakehouse, defining how data lands, is stored, retained, and governed across S3 and Snowflake/Databricks.

You will drive the architecture for bronze/silver/gold layers and manage CDC streaming with Kafka and Debezium, ensuring accuracy and regulatory compliance for millions of daily financial events. You will optimize partitioning, file layout, and query performance on Trino, Athena, and Snowflake, while enforcing data governance,

Qualifications

• 3+ years of data engineering with ownership of a large data lake/lakehouse.
• Cloud object storage (S3/GCS) plus Snowflake, Databricks, or BigQuery in production.
• Hands-on with Apache Iceberg/Delta/Hudi formats and open table standards.
• Deep knowledge of partitioning, file layout, and query optimization at multi-terabyte scales.
• Experience with CDC and streaming ingestion (Kafka + Debezium).

Responsibilities

• Own the lakehouse architecture across bronze/silver/gold layers and format choices.
• Land data via CDC streaming, handle late/duplicate events, ensure data freshness.
• Design data layout for speed and cost with partitioning and compaction.
• Manage retention, archiving, GDPR deletion, and immutable storage.
• Ensure governance: catalog, lineage, access control, PII masking, auditing.

Skills

Data lake architecture

SQL proficiency

Python

Data modeling

Data governance

GDPR/compliance

CDC/streaming ingestion

Cloud storage basics

Stakeholder ownership

Tools

Snowflake

Databricks

BigQuery

Iceberg

Delta Lake

Hudi

Kafka

Debezium

Trino

Athena

Airflow

Dagster

Job description

The Role

You'll own the data lake — the system of record for millions of financial events a day across million of active users. You decide how that data lands, is stored, retained, and governed on S3 + Snowflake/Databricks, so analytics, finance, and regulators all see accurate, reconciled data with zero drift from source.

What you'll be doing

• Own the lakehouse architecture: bronze/silver/gold layers, Iceberg/Delta tables, schema evolution.
• Land operational data via CDC streaming (Kafka, Debezium), handling late and duplicate events.
• Design data layout for speed and cost: partitioning, compaction, file sizing, query performance on Trino/Athena/Snowflake.
• Own retention and archival: storage tiering, regulatory retention, immutability, GDPR deletion.
• Guarantee correctness: freshness SLAs, drift detection, reconciliation against the source wallet and ledger systems.
• Own governance: catalog and lineage, row/column access control, PII masking, encryption, audit trails.
• Monitor ingestion health, data anomalies, and cloud storage/compute spend.

What we're looking for

• 3+ years in data engineering, with real ownership of a large-scale data lake or lakehouse.
• Cloud object storage (S3/GCS) plus Snowflake, Databricks, or BigQuery in production.
• Hands-on with an open table format (Iceberg, Delta, or Hudi).
• Deep understanding of partitioning, file layout, and query optimization at terabyte-plus scale.
• CDC and streaming ingestion experience (Kafka + Debezium or equivalent).
• Strong SQL and data modeling; enough relational grounding to reason about the OLTP systems you capture from — critical for financial ledgers.
• Data lake governance: catalogs, lineage, access control, PII, retention.
• Python and an orchestrator (Airflow/Dagster) as tools — this is an architecture and ownership role, not heads-down coding.
• Nice to have: fintech, iGaming, or another regulated, audit-heavy environment.