Senior Data Engineer - Real-Time Pipelines on GCP (Remote)

Hack-the-Box; Warszawa; Hybrid; PLN 389,000 - 606,000; Full time

Benefits offered by this job

Private health care

Paid paternity leave

25 annual leave days

Free lunch & snacks

€120 Ticket Restaurant

Job summary

Hack The Box is seeking a Senior Data Engineer to own and evolve data pipelines on GCP, building new ones and hardening existing ones. You will design ELT/ETL on BigQuery and ClickHouse, create real-time pipelines with Pub/Sub and Kafka, and orchestrate with Airflow while ensuring data quality for analytics and ML.

You will collaborate with ML engineers on feature pipelines, monitoring data drift, and retraining workflows, while treating infrastructure as code and integrating REST APIs and 3rd

Qualifications

• ,

Responsibilities

• Design batch and streaming pipelines on Dataflow, Pub/Sub, and Kafka feeding BigQuery, Bigtable, ClickHouse.
• Help migrate Snowflake to a GCP-native stack and retire shadow pipelines.
• Own the Airflow orchestration layer with SLAs, retries, and data quality gates.
• Model data for analytics and ML, including feature pipelines for training and online inference.
• Collaborate with ML engineers on feature stores, drift monitoring, and retraining workflows.

Skills

Data modelling

GCP data services

SQL

Python

Airflow

Data quality

Docker

Kubernetes

CI/CD

BigQuery

Tools

BigQuery

Pub/Sub

Dataflow

ClickHouse

Kafka

Airflow

dbt

Snowflake

Docker

Kubernetes

Job description

Hack The Box is seeking a Senior Data Engineer to own and evolve data pipelines on GCP, building new ones and hardening existing ones. You will design ELT/ETL on BigQuery and ClickHouse, create real-time pipelines with Pub/Sub and Kafka, and orchestrate with Airflow while ensuring data quality for analytics and ML.

You will collaborate with ML engineers on feature pipelines, monitoring data drift, and retraining workflows, while treating infrastructure as code and integrating REST APIs and 3rd