Senior Data Engineer – Scala / Spark / Hadoop (f/m)

B2Bnetwork; Warszawa; Hybrid; PLN 140,000 - 190,000; Full time

Job summary

B2Bnetwork is seeking an experienced Data Engineer to join an international team working on a strategic regulatory initiative within the financial sector. You will develop and maintain a data platform and reporting processes in a complex Big Data environment, using Hadoop, Spark, Kafka, Airflow and Jenkins.

You will collaborate with stakeholders across regions, design scalable pipelines, optimise SQL workflows and provide L3 production support while continuously improving data quality and

Qualifications

• Strong hands-on experience with Scala.
• Proven experience with Hadoop in a commercial setting.
• Practical knowledge of Spark and large-scale data processing.
• Experience building and maintaining Data Engineering pipelines.
• Experience with Kafka and event-driven architectures.
• Practical knowledge of CI/CD, especially Jenkins.
• Strong Git, code review and PR processes.
• Solid SQL development and optimisation skills.
• Experience supporting production environments.
• Ability to independently investigate complex technical issues.
• English proficiency for daily work in an international team.
• Willingness to work from the Warsaw or Gdańsk office twice per week.
• Ability to work from Poland.

Responsibilities

• Design, develop and maintain Data Engineering solutions using Hadoop and Spark.
• Build scalable solutions for processing large data volumes.
• Translate complex business requirements into technical solutions.
• Develop and optimise SQL queries and data-processing workflows.
• Build and enhance data pipelines and reporting processes.
• Create and maintain workflows using Apache Oozie and Apache Airflow.
• Develop data solutions using Scala.
• Work with event-driven architecture based on Apache Kafka.
• Maintain and improve CI/CD processes using Git, PRs and Jenkins.
• Provide L3 support for production environments.
• Investigate production incidents with operations teams.
• Collaborate with stakeholders to define requirements and analyse issues.
• Ensure quality, performance, security and reliability of data solutions.

Skills

Scala

Hadoop

Spark

Data pipelines

Kafka

Jenkins CI/CD

Git

SQL

Production support

English

Tools

Airflow

Job description

We are looking for an experienced Data Engineer to join an international team working on a strategic regulatory initiative within the financial sector. You will develop and maintain a data platform and reporting processes operating in a complex Big Data environment.

Responsibilities

• Design, develop and maintain Data Engineering solutions using Hadoop and Apache Spark.
• Build scalable solutions for processing large volumes of data.
• Analyse complex business requirements and translate them into effective technical solutions.
• Develop and optimise SQL queries and data-processing workflows.
• Build and enhance data pipelines and reporting processes.
• Create and maintain workflows using Apache Oozie and Apache Airflow.
• Develop data solutions using Scala.
• Work with event-driven architecture based on Apache Kafka.
• Maintain and improve CI/CD processes using Git, pull requests and Jenkins.
• Provide L3 support for production environments.
• Investigate production incidents and technical problems in cooperation with relevant operational teams.
• Collaborate with business stakeholders to define requirements and analyse issues.
• Ensure the quality, performance, security and reliability of data solutions.

Requirements

• Strong hands‑on experience with Scala.
• Commercial experience with Apache Hadoop.
• Practical knowledge of Apache Spark and large-scale data processing.
• Experience building and maintaining Data Engineering pipelines.
• Experience with Apache Kafka and event‑driven architecture.
• Practical knowledge of CI/CD processes, particularly Jenkins.
• Very good knowledge of Git, code review and pull request processes.
• Strong SQL development and optimisation skills.
• Experience supporting production environments.
• Ability to independently investigate complex technical issues.
• English proficiency sufficient for daily work in an international team.
• Availability to work from the Warsaw or Gdańsk office twice per week.
• Ability to perform the work from Poland.

Nice to have

• Kubernetes experience.
• Knowledge of Apache Flink.
• Experience with Apache Airflow or Apache Oozie.
• Previous involvement in regulatory or reporting projects.
• Experience in the financial sector.

What we offer

• Participation in a strategic regulatory initiative with a tangible impact on the financial sector.
• Work within a large and advanced Big Data environment.
• A modern technology stack including Scala, Hadoop, Spark, Kafka, Airflow and Jenkins.
• A high level of independence and influence over technical solutions.
• Cooperation with an experienced international team.
• Stable, long‑term cooperation.
• A professional environment focused on quality, security and expert development.
• An opportunity to gain experience in financial regulations and regulatory reporting.