Senior Cloud Data Engineer: Big Data Pipelines & Scale

Bonapolia; Warszawa; On-site; PLN 240,000 - 360,000; Full time

Job summary

Bonapolia is seeking a Senior Data Engineer to architect and implement high-volume batch and streaming data ingestion pipelines across AWS and GCP. You will drive data quality, lineage, and orchestration with Airflow while mentoring junior engineers.

You will collaborate with software, ML, and analytics teams to ship scalable data platforms and contribute to data lake and warehouse initiatives. This role requires 10+ years in data/web-scale environments.

Qualifications

• BS/MS in Computer Science with 10+ years of experience in distributed systems, data engineering, or software engineering.
• Strong coding skills in Python, Java, or Scala.
• Experience with SQL and NoSQL databases (BigQuery, Teradata, MySQL, Postgres, Cassandra).
• Experience with big data technologies: Hadoop, Hive, Apache Spark.
• Experience with Airflow or any other orchestration tool.
• In-depth experience with ETL, data lineage, data quality issues, backfills — ability to drive independent resolution and communication.
• Experience with batch and streaming data pipelines.
• Experience with GCP or AWS cloud technologies, particularly related to data processing at scale.
• Experience designing, implementing, and supporting production services with tight SLAs.
• Experience with central logging, metrics, monitoring, and alerting tools (Elasticsearch, Wavefront).
• Clear understanding of testing methodologies, CI/CD, and industry best practices.
• Strong coding skills in Python.
• Experience with Google Data Streams, Google Dataproc.
• Experience with Change Data Capture (CDC) technologies.
• Experience with modern data warehouse technology such as Delta Lake.

Responsibilities

• Design and develop high-volume batch and streaming data ingestion pipelines spanning AWS and GCP data platforms.
• Conceive, code, and launch the next-generation data ingestion and curation platforms.
• Participate in defining requirements and system and data architectural discussions.
• Technically lead and mentor junior engineers on best practices in software development and data engineering lifecycle.
• Collaborate with cross-functional agile teams of software engineers, data engineers, ML experts, and data analysts in building new product features.

Skills

Python

Java

Scala

SQL

NoSQL

ETL

Batch processing

Streaming data

CI/CD

Education

BS/MS in Computer Science

Tools

Airflow

Elasticsearch

Wavefront

GCP

AWS

Spark

Hadoop

Job description

Bonapolia is seeking a Senior Data Engineer to architect and implement high-volume batch and streaming data ingestion pipelines across AWS and GCP. You will drive data quality, lineage, and orchestration with Airflow while mentoring junior engineers.

You will collaborate with software, ML, and analytics teams to ship scalable data platforms and contribute to data lake and warehouse initiatives. This role requires 10+ years in data/web-scale environments.