Senior Data Engineer

Hack The Box; Greater London; Hybrid; GBP 60,000 - 80,000; Full time

Benefits offered by this job

Private health care

Paid paternity leave

25 annual leave days

Free lunch & snacks at the office

Training budget and conference participation

State-of-the-art equipment

Flexible WFH (Hybrid Model)

Job summary

Hack The Box is seeking a skilled Data Engineer to own and evolve our data pipelines on GCP. You will design, build, and maintain both batch and streaming data pipelines, ensuring data quality and reliability across the organization.

Ideal candidates should have strong skills in data modelling, production experience with GCP services, and proficiency in SQL and Python. We offer a hybrid work model with options for remote work or relocation to our Athens Tech Hub, alongside a variety of benefits including private health care and training budgets.

Qualifications

• Strong data modelling and warehouse architecture skills.
• Hands-on experience with GCP data services, especially BigQuery.
• Production experience with streaming pipelines.

Responsibilities

• Design and build batch and streaming pipelines on GCP.
• Own the orchestration layer in Airflow.
• Model data for analytics and ML.

Skills

Strong data modelling skills

GCP data services

Production experience with streaming pipelines

Solid SQL

Solid Python

Docker and Kubernetes experience

Workflow orchestration with Airflow

Tools

Dataflow

Pub/Sub

ClickHouse

Flink

Spark Structured Streaming

Job description

We are excited to welcome you to our innovative team where we redefine cyber security expertise standards by connecting business and community through engaging hacking experiences.

Core Mission

You will own and evolve our data pipelines on GCP—building new ones, hardening existing ones, improving data quality, and making clean, trustworthy data available across the organisation. You will work end‑to‑end on streaming and batch pipelines, from CDC and event ingestion through transformation, serving, and the feature layer that powers our ML and AI products.

Responsibilities

• Design and build batch and streaming pipelines on Dataflow, Pub/Sub, and Kafka feeding BigQuery, Bigtable, and ClickHouse.
• Help drive the migration off Snowflake onto our GCP native stack and retire legacy pipelines.
• Own the orchestration layer in Airflow, including SLAs, retries, and data quality gates.
• Model data for analytics and for ML—including feature pipelines that serve both training and low‑latency online inference.
• Partner with ML engineers on feature stores, drift monitoring and retraining workflows.
• Capture requirements from stakeholders and translate them into well‑scoped data products.
• Continuously improve data quality, reliability, observability and cost efficiency.
• Identify new data sources worth acquiring and integrate them cleanly.

Qualifications

• Strong data modelling and warehouse architecture skills (dimensional modelling, event‑driven, lakehouse patterns).
• Hands‑on experience with GCP data services—BigQuery is a must; Pub/Sub, Dataflow, Bigtable, Cloud Composer are strong pluses.
• Production experience with streaming pipelines on Dataflow/Beam, Flink, or Spark Structured Streaming, ingesting from Kafka and/or Pub/Sub.
• Solid SQL and strong Python—production‑quality code.
• Experience with ClickHouse or another columnar OLAP engine in production.
• Workflow orchestration experience with Airflow (or Prefect/Dagster).
• Comfortable with dbt or equivalent transformation frameworks.
• Experience migrating off legacy warehouses (Snowflake, Redshift, Synapse) onto cloud‑native stacks.
• Working knowledge of ML in production—feature engineering, feature stores, model deployment, drift monitoring, retraining.
• Docker and Kubernetes experience.
• CI/CD mindset, infrastructure‑as‑code sensibility and a bias for simple, observable systems.
• Bonus: CDC tooling (Datastream, Debezium), Vertex AI / Feature Store.

Benefits

• Private health care.
• Paid paternity leave.
• 25 annual leave days.
• Free lunch & snacks at the office.
• Ticket Restaurant by Edenred.
• Dedicated budget for training and professional development, participation in conferences.
• Full access to our lab offerings for learning how to hack.
• State‑of‑the‑art equipment (Mac, iPhone, mobile plan).
• Flexible WFH (Hybrid Model) – fully remote also available if not in Athens.

Location & Work Mode

Europe/Greece. Choose between fully remote work across Europe or relocating to our Athens Tech Hub with relocation support.

Relocation bonuses: 10% relocation bonus on the annual salary; 50% tax reduction; accommodation support for the first few weeks.

Company Culture and Diversity Statement

At Hack The Box we are committed to fostering a diverse, inclusive, and equitable workplace. We believe that diversity enriches our performance, services, and the communities we serve. All applications are considered solely based on merit, skills, and qualifications. We do not discriminate on grounds of race, color, religion, gender, gender identity or expression, sexual orientation, national origin, genetics, disability, age, or veteran status. We are dedicated to providing a fair and respectful work environment that reflects our values.