Data Platform Engineer

Software Mind

Warszawa

On-site

PLN 180,000 - 300,000

Full time

2 days ago

Be an early applicant

Get more replies from employers

Send a job-specific resume in minutes.

Benefits offered by this job

Flexible remote work

International projects

Language classes

Private healthcare

Multisport card

Job summary

Software Mind is seeking an experienced Data/Platform Engineer to join our international teams. You will build and operate CDC pipelines from PostgreSQL into Azure, using Kafka Connect and Debezium, and run stateful workloads on AKS.

You’ll automate with Python, integrate with Azure data services, and apply IaC practices to ensure reproducibility, security, and reliable delivery in a fast-paced environment.

Qualifications

Hands-on experience with streaming or CDC pipelines.
Kafka knowledge with Connect deployment.
Strong SQL and PostgreSQL including replication concepts.
Understanding CDC concepts and event ordering.
Python automation and testing.
Experience with Azure data services such as Event Hubs/ADLS/Azure PostgreSQL.
Kubernetes usage and troubleshooting in production.

Responsibilities

Build and operate CDC pipelines from PostgreSQL into Azure using Kafka Connect and Debezium.
Configure, deploy and scale connectors end to end including offsets and snapshot strategy.
Run pipelines as stateful workloads on Kubernetes (AKS) with configuration and secrets management.
Monitor and troubleshoot production platform: failures, rebalances, throughput, retries and recovery.
Automate platform tasks in Python for onboarding data sources and testing.
Integrate CDC with Azure data stack (Event Hubs, ADLS, Azure PostgreSQL, ADF, Databricks).
Manage infrastructure as code to keep environments reproducible and auditable.
Apply data protection controls for sensitive data (masking, hashing, retention).

Skills

Kafka

PostgreSQL

Python

Azure

Kubernetes

Debezium

Data pipelines

CI/CD

Tools

AKS

Terraform

Databricks

ADF

Azure Event Hubs

Bicep

Git

Job description

Software Mind develops solutions that make an impact for companies around the globe. Tech giants & unicorns, transformative projects, emerging technologies and limitless opportunities - these are a few words that describe an average day for us. Building cross-functional engineering teams that take ownership and crave more means we're always on the lookout for talented people who bring passion and creativity to every project. Our culture embraces openness, acts with respect, shows grit & guts and combines employment with enjoyment.

Job Description

Project – the aim you'll have

Our customer provides innovative solutions and insights that enable our clients to manage risk and hire the best talent. Their advanced global technology platform supports fully scalable, configurable screening programs that meet the unique needs of over 33,000 clients worldwide. Headquartered in Atlanta, GA, they have an internationally distributed workforce spanning 19 countries with about 5,500 employees. Our partner perform over 93 million screens annually in over 200 countries and territories.

Position – how you'll contribute

Build and operate change-data-capture pipelines from PostgreSQL into Azure, using Kafka Connect and Debezium as the core of the platform
Configure, deploy and scale connectors end to end - connector setup, task management, offsets, schema history, and snapshot strategy
Run these pipelines as stateful workloads on Kubernetes (AKS), covering configuration, secrets, networking and resource tuning
Monitor and troubleshoot the platform in production: connector failures, task rebalances, restarts, throughput and backpressure, message-size limits, retries and recovery
Automate the platform in Python - configuration-driven onboarding of new data sources, pipeline orchestration, monitoring and alerting, recovery workflows, and automated testing
Integrate CDC streams with the wider Azure data stack: Event Hubs, ADLS, Azure PostgreSQL, ADF and Databricks
Manage platform infrastructure as code, so environments are reproducible and changes are reviewable
Apply data protection requirements to sensitive data flowing through the pipelines - masking, hashing, access control and retention

Qualifications

Expectations – the experience you need

Solid commercial experience as a data or platform engineer, with hands‑on work on streaming or CDC pipelines rather than batch reporting alone
Practical Kafka knowledge - topics, partitions, offsets, consumer groups and delivery semantics - including at least one Kafka Connect deployment you ran yourself
Strong SQL and PostgreSQL skills, with working knowledge of WAL, logical replication, replication slots and replication lag
Working understanding of CDC concepts: initial snapshots, inserts, updates and deletes, event ordering, at‑least‑once delivery, and schema evolution
Confident Python for automation and tooling - orchestration, monitoring, recovery scripts, and automated tests
Hands‑on experience with Azure data services, for example Event Hubs, ADLS or Azure PostgreSQL
Comfortable working with Kubernetes as a user: deploying workloads, handling configuration and secrets, reading logs, debugging failing pods
Ability to debug a running pipeline from metrics and logs - telling throughput problems from backpressure, retries or a genuine connector failure

Additional skills – the edge you have

Production experience with Debezium specifically - snapshot strategies on large tables, schema history recovery, offset loss, and bringing connectors back after failure
Experience operating stateful workloads on AKS: StatefulSets, stable worker identity, and resource tuning under load
Infrastructure-as-code and CI/CD for data platform components (Terraform, Bicep or similar)
Hands‑on work with Databricks and ADF at production scale
Experience implementing data protection controls for sensitive data - masking, hashing, access control and retention policies

Additional Information

Our offer – professional development, personal growth