Senior Data Engineer (Data Unification)

Bonapolia; Warszawa; On-site; PLN 240,000 - 360,000; Full time

Job summary

Bonapolia is seeking a Senior Data Engineer to architect and implement high-volume batch and streaming data ingestion pipelines across AWS and GCP. You will drive data quality, lineage, and orchestration with Airflow while mentoring junior engineers.

You will collaborate with software, ML, and analytics teams to ship scalable data platforms and contribute to data lake and warehouse initiatives. This role requires 10+ years in data/web-scale environments.

Qualifications

• BS/MS in Computer Science with 10+ years of experience in distributed systems, data engineering, or software engineering.
• Strong coding skills in Python, Java, or Scala.
• Experience with SQL and NoSQL databases (BigQuery, Teradata, MySQL, Postgres, Cassandra).
• Experience with big data technologies: Hadoop, Hive, Apache Spark.
• Experience with Airflow or any other orchestration tool.
• In-depth experience with ETL, data lineage, data quality issues, backfills — ability to drive independent resolution and communication.
• Experience with batch and streaming data pipelines.
• Experience with GCP or AWS cloud technologies, particularly related to data processing at scale.
• Experience designing, implementing, and supporting production services with tight SLAs.
• Experience with central logging, metrics, monitoring, and alerting tools (Elasticsearch, Wavefront).
• Clear understanding of testing methodologies, CI/CD, and industry best practices.
• Strong coding skills in Python.
• Experience with Google Data Streams, Google Dataproc.
• Experience with Change Data Capture (CDC) technologies.
• Experience with modern data warehouse technology such as Delta Lake.

Responsibilities

• Design and develop high-volume batch and streaming data ingestion pipelines spanning AWS and GCP data platforms.
• Conceive, code, and launch the next-generation data ingestion and curation platforms.
• Participate in defining requirements and system and data architectural discussions.
• Technically lead and mentor junior engineers on best practices in software development and data engineering lifecycle.
• Collaborate with cross-functional agile teams of software engineers, data engineers, ML experts, and data analysts in building new product features.

Skills

Python

Java

Scala

SQL

NoSQL

ETL

Batch processing

Streaming data

CI/CD

Education

BS/MS in Computer Science

Tools

Airflow

Elasticsearch

Wavefront

GCP

AWS

Spark

Hadoop

Job description

Tech Level: Senior
Employment type: Full time
Candidate Location: EU, Georgia, Uzbekistan
Start: ASAP
Project Phase: Ongoing

Customer Description:

The client is an experienced marketplace that brings people more ways to get the most out of their city or wherever they may be. By enabling real-time mobile commerce across local businesses, live events, and travel destinations, the platform helps people find and discover experiences that make for a full, fun, and rewarding life. With thousands of employees spread across multiple continents, the company maintains a culture that inspires innovation, rewards risk-taking, and celebrates success.

Project Description:

The Data team is at the heart of all things data, working on defining and building the next-generation cloud-based solutions to ingest and curate petabytes of data into a data lake and data warehouse. The mission is to empower data analysts and data scientists across all business units to make informed business decisions. This role offers a unique combination of skills in distributed systems, big data, and scalable high-performance production systems.

Hard Skills / Must Have:

BS/MS in Computer Science with 10+ years of experience in distributed systems, data engineering, or software engineering.

Strong coding skills in Python, Java, or Scala.

Experience with SQL and NoSQL databases (BigQuery, Teradata, MySQL, Postgres, Cassandra).

Experience with big data technologies: Hadoop, Hive, Apache Spark.

Experience with Airflow or any other orchestration tool.

In-depth experience with ETL, data lineage, data quality issues, backfills — ability to drive independent resolution and communication.
Experience with batch and streaming data pipelines.

Experience with GCP or AWS cloud technologies, particularly related to data processing at scale.

Experience designing, implementing, and supporting production services with tight SLAs.

Experience with central logging, metrics, monitoring, and alerting tools (Elasticsearch, Wavefront).

Clear understanding of testing methodologies, CI/CD, and industry best practices.

Hard Skills / Nice to Have:

Strong coding skills in Python.

Experience with Google Data Streams, Google Dataproc.

Experience with Change Data Capture (CDC) technologies.

Experience with modern data warehouse technology such as Delta Lake.

Responsibilities

Design and develop high-volume batch and streaming data ingestion pipelines spanning AWS and GCP data platforms.

Conceive, code, and launch the next-generation data ingestion and curation platforms.

Participate in defining requirements and system and data architectural discussions.

Technically lead and mentor junior engineers on best practices in software development and data engineering lifecycle.

Collaborate with cross-functional agile teams of software engineers, data engineers, ML experts, and data analysts in building new product features.

Technology Stack

Cloud: GCP, AWS.

Big Data: Apache Spark, Hadoop, Hive.

Databases: BigQuery, Teradata, MySQL, Postgres, Cassandra.

Orchestration: Apache Airflow.

Monitoring & Logging: Elasticsearch, Wavefront.

CI/CD: industry-standard tools and practices.