Senior Data Engineer | GCP | Kafka | BigQuery | Python

Talentedge; Kraków; On-site; PLN 180,000 - 240,000; Full time

Job summary

Talentedge is seeking a Senior Data Engineer to join its global Data Engineering function and drive a major transformation of its data platform. You will design, build and operate production-grade data pipelines across both batch and real-time environments on GCP.

You will work with BigQuery, Airflow, Kafka, dbt and a Medallion architecture to enable analytics, compliance, Data Science and AI/ML use cases at scale.

Qualifications

• Strong commercial experience building and operating production data pipelines on GCP.
• Proficient in SQL and Python.
• Experience with Apache Kafka or comparable streaming technologies.
• Experience with BigQuery, Airflow/Cloud Composer and GCS.
• Experience with dbt and modern data transformation approaches.
• Understanding of Medallion architecture and data platform best practices.
• Experience with data contracts, lineage, observability and pipeline SLOs.
• Experience working within a structured or regulated environment.

Responsibilities

• Designing and building scalable batch and real-time data pipelines.
• Developing Kafka-based event streams and Airflow-orchestrated workflows.
• Building transformations and data models in BigQuery.
• Developing and maintaining ingestion integrations and connectors.
• Implementing data quality controls, observability and pipeline monitoring.
• Contributing to engineering standards, code reviews and CI/CD practices.
• Supporting Data Science and AI-ready data initiatives.
• Mentoring junior engineers and contributing to the development of the wider engineering team.

Skills

GCP

SQL

Python

Apache Kafka

Airflow

dbt

BigQuery

Medallion architecture

Data quality

Observability

Tools

Cloud Composer

GCS

Cloud Run

Kafka

Job description

A leading FTSE 100 fintech is looking for a Senior Data Engineer to join its global Data Engineering function and contribute to a major transformation of its data platform.

The team is consolidating a complex data estate onto GCP, creating a scalable and trusted platform that supports analytics, compliance, Data Science and increasingly sophisticated AI/ML use cases.

This is a hands-on role suited to an experienced engineer who enjoys solving complex data problems and wants to have a meaningful influence on engineering standards and platform design.

The role

Working within a Data Engineering squad, you will design, build and operate production-grade data pipelines across both batch and real-time environments.

You will work across:

• GCP – BigQuery, Cloud Composer/Airflow, GCS and Cloud Run
• Apache Kafka – event-driven ingestion and real-time streaming
• Python and SQL – building reliable, production-grade data solutions
• dbt – transformation and data modelling
• Medallion architecture – Bronze, Silver and Gold layers
• Data quality and observability – contracts, lineage, monitoring and SLOs

Role responsibilities

• Designing and building scalable batch and real-time data pipelines
• Developing Kafka-based event streams and Airflow-orchestrated workflows
• Building transformations and data models in BigQuery
• Developing and maintaining ingestion integrations and connectors
• Implementing data quality controls, observability and pipeline monitoring
• Contributing to engineering standards, code reviews and CI/CD practices
• Supporting Data Science and AI-ready data initiatives
• Mentoring junior engineers and contributing to the development of the wider engineering team

Role Requirements

• Strong commercial experience building and operating production data pipelines on GCP
• Strong SQL and Python skills
• Solid experience with Apache Kafka or comparable streaming technologies
• Experience with BigQuery, Airflow/Cloud Composer and GCS
• Experience with dbt and modern data transformation approaches
• Understanding of Medallion architecture and data platform best practices
• Experience with data contracts, lineage, observability and pipeline SLOs
• Experience working within a structured or regulated environment

This is an opportunity to work on large-scale data engineering challenges within a global fintech, with the technical scope to make a real impact on the platform and how data is used across the organisation.