Senior Data Engineer - Real Time Streams

Globaldev Group; Poland; On-site; PLN 230,000 - 320,000; Full time

Benefits offered by this job

20 days paid vacation

5 sick days

Flexible schedule

Opportunity to influence product and技n

Professional growth and learning

Supportive team

Job summary

Globaldev Group is seeking a senior software/data engineer to own the real-time data platform end-to-end. You will design, implement, deploy, and operate Flink streaming pipelines, CDC pipelines, and Kafka Connect bridges that feed Postgres/TimescaleDB/ClickHouse/BigQuery.

You will troubleshoot production issues, ensure data quality, and mentor junior engineers. The role emphasizes independent ownership, strong Java expertise, and hands-on experience with streaming architectures in a Kubernetes

Qualifications

• 5+ years of professional software/data engineering experience.
• Strong Java expertise and production experience with Flink streaming pipelines.
• Deep Kafka experience including Connect, consumer groups and schema management.
• Hands-on CDC experience with Debezium or Flink CDC.
• Experience designing data pipelines from Kafka through Flink to databases/warehouses.

Responsibilities

• Design, implement, and operate stateful Apache Flink streaming pipelines with advanced features.
• Build and maintain CDC pipelines moving operational DB changes into the streaming platform.
• Develop and operate Kafka and Kafka Connect pipelines, with schema management and DLQ routing.
• Own the end-to-end path from Kafka to downstream sinks (Postgres/TimescaleDB/ClickHouse/BigQuery).
• Troubleshoot production issues (backpressure, checkpoint failures, latency, sink performance) and contribute to deployments (Helm, ArgoCD, GKE).
• Mentor mid-level engineers and participate in architectural decisions.

Skills

Java

Independent worker

Analytical thinking

Tools

Flink

Kafka

Debezium

Kubernetes

PostgreSQL

TimescaleDB

ClickHouse

BigQuery

Job description

You will be a senior engineer on the team that owns the real-time data platform. The platform turns operational events (orders, driver locations, geofence transitions, shifts, MQTT session events) into live analytics, routing inputs, and warehoused data across Postgres/TimescaleDB, ClickHouse, and BigQuery. You will own the platform end-to-end: design, implementation, deployment, and production operation of the streaming jobs, CDC pipelines, Kafka Connect bridges, and downstream sinks that move this data.

Responsibilities

• Designing, implementing, and operating stateful Apache Flink streaming pipelines: keyed state, windowing, watermarks, timers, side outputs, custom sources/sinks
• Designing, developing, deploying, and maintaining Change Data Capture (CDC) pipelines using Flink CDC or Debezium, moving operational database changes into the streaming platform with correct snapshot/incremental handling, schema evolution, and downstream idempotency
• Building and operating Kafka and Kafka Connect pipelines: topic and partition design, source/sink connectors in distributed mode, schema and converter management, and deadletter routing
• Owning the path from Kafka → Flink → Postgres / TimescaleDB / ClickHouse / BigQuery, including schema design, idempotency strategy, batch tuning, and observability
• Diagnosing and fixing production issues: checkpoint failures, backpressure, state growth, sink slowness, autoscaler oscillation, restart loops
• Hardening the platform: delivery guarantees, watermarks, DLQ handling, schema migrations, alerting coverage. Code reviews and mentoring mid-level engineers
• Contributing to the deployment side: Helm charts, ArgoCD applications, GKE configuration, Grafana dashboards, Prometheus alert rules
• Influencing direction: state backend choices, schema migrations, when a pipeline needs to be split or rebuilt

Requirements

• 5+ years of professional software/data engineering experience, with strong Java expertise
• Production experience with Apache Flink and stateful real-time streaming pipelines
• Strong Apache Kafka experience, including Kafka Connect, consumer groups, partitions, delivery guarantees, and schema management
• Hands-on CDC experience using Debezium or Flink CDC
• Experience designing and operating Kafka → Flink → database/data warehouse pipelines in production
• Strong understanding of PostgreSQL and experience with at least one analytical database such as ClickHouse or BigQuery
• Experience with Kubernetes and deploying/operating production data workloads
• Proven ability to troubleshoot and optimize production streaming systems—backpressure, checkpoint failures, state growth, latency, lag, and sink performance
• Strong understanding of data consistency, idempotency, schema evolution, and observability
• Ability to work independently and own a data platform component end-to-end, from design through production operation

Will be a plus

• Experience with TimescaleDB, hypertables, and time-series data
• Experience with PostGIS and geospatial/spatial query optimization
• Deep ClickHouse performance tuning, including MergeTree and partitioning strategies
• Strong BigQuery optimization and data modeling experience
• Experience with Flink Kubernetes Operator and Flink autoscaler tuning
• Experience with ArgoCD, Helm, GKE, Prometheus, and Grafana
• Experience designing high-throughput, low-latency real-time systems at scale
• Experience with MQTT or IoT/event-driven systems
• Knowledge of Kafka Schema Registry, Avro/Protobuf, and advanced schema evolution
• Experience mentoring engineers and leading technical decisions around streaming architecture
• Experience with routing, logistics, delivery, mobility, or location-based platforms would be particularly relevant

What We Offer

• 20 days of paid vacation, 5 sick days, and public holidays
• Flexible schedule with a high level of autonomy
• Opportunity to influence product and technical decisions
• Professional growth and learning opportunities
• Comfortable working environment with a supportive team
• Mobilization reservation available upon successful completion of the probation period (for Ukrainian colleagues)