Data Engineer: GCP Pipelines & AI-Ready Data

IG KnowHow; Kraków; Hybrid; PLN 180,000 - 260,000; Full time

Benefits offered by this job

Private medical cover

Home office equipment reimbursement

Performance related bonus

LinkedIn Learning platform free access

Three additional days off a year

Job summary

IG Group is seeking a Data Engineer to design and operate production data pipelines in a hybrid model. You will own ingestion, transformation, and data quality across the analytics, compliance, and AI-ready data platforms, contributing to a scalable, self-service data environment.

You'll partner with a squad led by the Data Engineering Team Lead, tackle complex ingestion challenges, and help raise the bar for data governance and reliability across the firm.

Qualifications

• Proven hands-on experience building production data pipelines on GCP.
• Strong SQL and Python skills with real-time and batch processing.
• Familiar with data contracts, lineage, and observability in production.

Responsibilities

• Design and operate batch and real-time data pipelines on GCP with Kafka, BigQuery, and Airflow.
• Own data quality, contracts, observability, and SLA adherence.
• Develop in-house connectors and third-party integrations for ingestion stability.
• Contribute to standards, code reviews, and CI/CD for shared datasets.
• Support feature engineering and ML data pipelines with lineage and reproducibility.

Skills

BigQuery

Python

SQL

Airflow

Kafka

GCP

Tools

dbt

Cloud Composer

Airflow

Cloud Run

GCS

Job description

IG Group is seeking a Data Engineer to design and operate production data pipelines in a hybrid model. You will own ingestion, transformation, and data quality across the analytics, compliance, and AI-ready data platforms, contributing to a scalable, self-service data environment.

You'll partner with a squad led by the Data Engineering Team Lead, tackle complex ingestion challenges, and help raise the bar for data governance and reliability across the firm.