Poland-Based Data Lakehouse Engineer | Spark, CI/CD

B3 Consulting Poland; Polska; On-site; PLN 180,000 - 240,000; Full time

Benefits offered by this job

Health and sports benefits

Speed Coaching – short sessions fornew

Sports training

Team events and meetups

Job summary

B3 Consulting Poland is seeking a Data Lakehouse Developer to design and maintain scalable data lakehouse architectures using Iceberg, Delta Lake, or Apache Hudi, from ingestion to presentation layers.

You will build PySpark pipelines and collaborate in Agile teams to ensure robust data delivery, quality, and performance. The role emphasizes CI/CD automation on Kubernetes and close collaboration with stakeholders in a professional, autonomous environment.

Qualifications

• 4+ years of hands-on production experience in data engineering.
• Strong proficiency in Python, PySpark, Spark-SQL, DataFrames and Structured Streaming.
• Proven experience with MongoDB and Apache Iceberg (versioning, upserts/merges).
• CI/CD automation experience (Jenkins) and containerized deployments on Kubernetes.
• You need to live in Poland and/or be eligible to work here.

Responsibilities

• Data Lakehouse & Architecture: Design and maintain versioned data layers (Bronze, Silver, Gold) using Iceberg/Delta/Lake or Hudi, including CI/CD for schema evolution.
• Spark & MongoDB Integration: Build PySpark/Spark-SQL pipelines for data reconciliation and optimized reads.
• Performance & Cost Tuning: Optimize Spark jobs to reduce latency and processing costs.
• Quality, Monitoring & SLAs: Implement data quality checks and monitoring for batch/streaming pipelines.
• Kubernetes & Automation: Automate Spark deployments on Kubernetes using Docker, Spark Operator, CI/CD.
• Delivery & Collaboration: Conduct code reviews and collaborate in Agile teams.

Skills

Python

PySpark

Spark-SQL

DataFrames

Structured Streaming

Tools

Jenkins

Kubernetes

Docker

Spark Operator

Job description

B3 Consulting Poland is seeking a Data Lakehouse Developer to design and maintain scalable data lakehouse architectures using Iceberg, Delta Lake, or Apache Hudi, from ingestion to presentation layers.

You will build PySpark pipelines and collaborate in Agile teams to ensure robust data delivery, quality, and performance. The role emphasizes CI/CD automation on Kubernetes and close collaboration with stakeholders in a professional, autonomous environment.