Global Data Platform Engineer – Scala/Spark/Kafka/Hadoop

B2Bnetwork; Warszawa; Hybrid; PLN 140,000 - 190,000; Full time

Job summary

B2Bnetwork is seeking an experienced Data Engineer to join an international team working on a strategic regulatory initiative within the financial sector. You will develop and maintain a data platform and reporting processes in a complex Big Data environment, using Hadoop, Spark, Kafka, Airflow and Jenkins.

You will collaborate with stakeholders across regions, design scalable pipelines, optimise SQL workflows and provide L3 production support while continuously improving data quality and

Qualifications

• Strong hands-on experience with Scala.
• Proven experience with Hadoop in a commercial setting.
• Practical knowledge of Spark and large-scale data processing.
• Experience building and maintaining Data Engineering pipelines.
• Experience with Kafka and event-driven architectures.
• Practical knowledge of CI/CD, especially Jenkins.
• Strong Git, code review and PR processes.
• Solid SQL development and optimisation skills.
• Experience supporting production environments.
• Ability to independently investigate complex technical issues.
• English proficiency for daily work in an international team.
• Willingness to work from the Warsaw or Gdańsk office twice per week.
• Ability to work from Poland.

Responsibilities

• Design, develop and maintain Data Engineering solutions using Hadoop and Spark.
• Build scalable solutions for processing large data volumes.
• Translate complex business requirements into technical solutions.
• Develop and optimise SQL queries and data-processing workflows.
• Build and enhance data pipelines and reporting processes.
• Create and maintain workflows using Apache Oozie and Apache Airflow.
• Develop data solutions using Scala.
• Work with event-driven architecture based on Apache Kafka.
• Maintain and improve CI/CD processes using Git, PRs and Jenkins.
• Provide L3 support for production environments.
• Investigate production incidents with operations teams.
• Collaborate with stakeholders to define requirements and analyse issues.
• Ensure quality, performance, security and reliability of data solutions.

Skills

Scala

Hadoop

Spark

Data pipelines

Kafka

Jenkins CI/CD

Git

SQL

Production support

English

Tools

Airflow

Job description

B2Bnetwork is seeking an experienced Data Engineer to join an international team working on a strategic regulatory initiative within the financial sector. You will develop and maintain a data platform and reporting processes in a complex Big Data environment, using Hadoop, Spark, Kafka, Airflow and Jenkins.

You will collaborate with stakeholders across regions, design scalable pipelines, optimise SQL workflows and provide L3 production support while continuously improving data quality and