Data Engineer — Data Lakehouse

Commit; Warszawa; Hybrid; PLN 150,000 - 220,000; Full time

Job summary

Commit in Kraków, Poland seeks a Data Engineer who will own the company data lake—system of record for millions of financial events daily. You decide how data lands, is stored, retained and governed on S3, Snowflake and Databricks, ensuring accurate, reconciled data with zero drift from source.

You will build a robust lakehouse, enable streaming ingestion, optimize partitioning and query performance, and implement governance and retention to support analytics, finance and regulators.

Qualifications

• 3 years' experience in hands-on delivery within lakehouse architecture, pipelines and ingestion.
• Lakehouse architecture with bronze/silver/gold layers and open table formats (Iceberg/Delta/Hudi).
• Data layout & query optimization at TB+ scale; partitioning, compaction, file sizing, and query performance on Trino/Athena/Snowflake.

Responsibilities

• Own the lakehouse architecture and data tiers.
• Land operational data via CDC streaming (Kafka, Debezium) and handle late/duplicate events.
• Design data layouts for speed and cost; optimize partitioning and file sizing.

Skills

SQL

Lakehouse

CDC streaming

Data modeling

Snowflake

Databricks

Python

Airflow

Partitioning

Trino/Athena

Tools

Iceberg

Delta

Hudi

Kafka

Debezium

S3

GCS

Snowflake

Databricks

Athena

Job description

We are looking for Data Engineer in Kraków, Poland who will own the company data lake — the system of record for millions of financial events a day (bets, wallet movements, live odds) across 12M+ active users. You decide how that data lands, is stored, retained, and governed on S3 + Snowflake/Databricks, so analytics, finance, and regulators all see accurate, reconciled data with zero drift from source.

Domain:

Regulated iGaming / wallet & ledger data. Audit-heavy: regulators, finance and analytics all consume the same tables. Millions of financial events per day, terabyte-plus scale.

What you'll be doing:

• Own the lakehouse architecture: bronze/silver/gold layers, Iceberg/Delta tables, schema evolution.
• Land operational data via CDC streaming (Kafka, Debezium), handling late and duplicate events.
• Design data layout for speed and cost: partitioning, compaction, file sizing, query performance on Trino/Athena/Snowflake.
• Own retention and archival: storage tiering, regulatory retention, immutability, GDPR deletion.
• Guarantee correctness: freshness SLAs, drift detection, reconciliation against the source wallet and ledger systems.
• Own governance: catalog and lineage, row/column access control, PII masking, encryption, audit trails.
• Monitor ingestion health, data anomalies, and cloud storage/compute spend.

Requirements:

Must-have:

• 3 years' experience in hands-on delivery within that architecture — pipelines, ingestion, models, monitoring
• Lakehouse architecture — bronze/silver/gold layering, an open table format (Iceberg, Delta, or Hudi), schema evolution.
• Data layout & query optimization at TB+ scale — partitioning, compaction, file sizing, query performance on Trino/Athena/Snowflake.
• Cloud lakehouse/DWH in production — Snowflake, Databricks, or BigQuery.
• CDC & streaming ingestion — Kafka + Debezium or equivalent; late, duplicate and out-of-order events.
• Strong SQL and data modeling — enough relational grounding to reason about the OLTP systems you capture from. Critical for financial ledgers.
• Correctness — freshness SLAs, drift detection, reconciliation against source wallet/ledger systems.
• Governance — catalogs, lineage, row/column access control, PII masking, retention, GDPR deletion.
• Cloud object storage — S3 or GCS, plus storage tiering and archival.
• Python and an orchestrator — Airflow or Dagster, as tools.

Location & work model:

Kraków, Poland. Hybrid — 2 days per week from the office.