Senior Data Engineer – Remote, GCP Pipelines Lead

Hack-the-Box; North Carolina; On-site; USD 140,000 - 160,000; Full time

Benefits offered by this job

Medical, Dental & Vision

401K with employer match

Home Office Allowance

Training & conferences

State‑of‑the‑art equipment

Job summary

Hack The Box is seeking a Senior Data Engineer to own and evolve data pipelines on GCP, building ELT/ETL processes and ensuring clean, trustworthy data for analytics, ML training, and online inference.

You’ll work end-to-end with Pub/Sub, Kafka, Dataflow, BigQuery, Bigtable, and ClickHouse, collaborating with ML engineers and migrating from Snowflake to a pure GCP-native stack. Remote US-based role with strong growth opportunities.

Qualifications

• Strong data modelling and warehouse architecture.
• Experience with GCP data services (BigQuery, Pub/Sub, Dataflow).
• Production experience with streaming pipelines (Dataflow, Flink, Spark).
• Solid SQL and Python production code.
• Experience with ClickHouse or other OLAP engines in production.
• Airflow/workflow orchestration experience.
• Comfortable with dbt or similar transformation frameworks.
• Experience migrating from legacy warehouses to cloud-native stacks.
• ML in production knowledge (feature stores, drift monitoring).
• Docker & Kubernetes experience.
• CI/CD mindset and infrastructure-as-code.
• Bonus: CDC tooling and Vertex AI/Feature Store.

Responsibilities

• Design and build batch and streaming pipelines on Dataflow, Pub/Sub, and Kafka feeding BigQuery, Bigtable, and ClickHouse.
• Help migrate from Snowflake to a GCP-native stack and retire shadow pipelines.
• Own the Airflow orchestration layer with SLAs and data quality gates.
• Model data for analytics and ML feature pipelines for training and low-latency inference.
• Collaborate with ML engineers on feature stores and retraining workflows.
• Translate stakeholder requirements into pragmatic data products.
• Improve data quality, observability, and cost efficiency.
• Identify and integrate new data sources.

Skills

Data modelling

GCP data services

Streaming pipelines

Airflow

ClickHouse

SQL

Python

Docker & Kubernetes

Data quality

ETL/ELT design

Tools

BigQuery

Dataflow

Pub/Sub

Kafka

Flink

Spark

Airflow

dbt

Docker

Kubernetes

Snowflake

Job description

Hack The Box is seeking a Senior Data Engineer to own and evolve data pipelines on GCP, building ELT/ETL processes and ensuring clean, trustworthy data for analytics, ML training, and online inference.

You’ll work end-to-end with Pub/Sub, Kafka, Dataflow, BigQuery, Bigtable, and ClickHouse, collaborating with ML engineers and migrating from Snowflake to a pure GCP-native stack. Remote US-based role with strong growth opportunities.