Senior Data Engineer: Real-Time Pipelines on GCP & Kafka

Talentedge; Kraków; On-site; PLN 180,000 - 240,000; Full time

Job summary

Talentedge is seeking a Senior Data Engineer to join its global Data Engineering function and drive a major transformation of its data platform. You will design, build and operate production-grade data pipelines across both batch and real-time environments on GCP.

You will work with BigQuery, Airflow, Kafka, dbt and a Medallion architecture to enable analytics, compliance, Data Science and AI/ML use cases at scale.

Qualifications

• Strong commercial experience building and operating production data pipelines on GCP.
• Proficient in SQL and Python.
• Experience with Apache Kafka or comparable streaming technologies.
• Experience with BigQuery, Airflow/Cloud Composer and GCS.
• Experience with dbt and modern data transformation approaches.
• Understanding of Medallion architecture and data platform best practices.
• Experience with data contracts, lineage, observability and pipeline SLOs.
• Experience working within a structured or regulated environment.

Responsibilities

• Designing and building scalable batch and real-time data pipelines.
• Developing Kafka-based event streams and Airflow-orchestrated workflows.
• Building transformations and data models in BigQuery.
• Developing and maintaining ingestion integrations and connectors.
• Implementing data quality controls, observability and pipeline monitoring.
• Contributing to engineering standards, code reviews and CI/CD practices.
• Supporting Data Science and AI-ready data initiatives.
• Mentoring junior engineers and contributing to the development of the wider engineering team.

Skills

GCP

SQL

Python

Apache Kafka

Airflow

dbt

BigQuery

Medallion architecture

Data quality

Observability

Tools

Cloud Composer

GCS

Cloud Run

Kafka

Job description

Talentedge is seeking a Senior Data Engineer to join its global Data Engineering function and drive a major transformation of its data platform. You will design, build and operate production-grade data pipelines across both batch and real-time environments on GCP.

You will work with BigQuery, Airflow, Kafka, dbt and a Medallion architecture to enable analytics, compliance, Data Science and AI/ML use cases at scale.