Big Data Engineer (Python/Spark) for Banking Data

Integral Solutions - Informatica Distributor; Warszawa; Hybrid; PLN 200,880 - 279,000; Full time

Job summary

A leading technology distributor in Warsaw is seeking a skilled Big Data Engineer to join an exciting project for a top banking client. You will build and enhance Big Data solutions using Python and Apache Spark. Ideal candidates have a minimum of 3 years' experience with BigData technologies and advanced SQL knowledge. The position requires working on-site once every two weeks, offering a blend of innovative environments and technical challenges.

Qualifications

• Minimum 3 years of experience in Python or Scala programming.
• Commercial experience with BigData technologies; Spark is essential.
• Advanced knowledge of SQL and integrating data from multiple data sources.

Responsibilities

• Write and maintain data pipelines and applications using Python.
• Work with Apache Spark to process large datasets.
• Combine data from different sources into a single system.

Skills

Python

Apache Spark

SQL

Data Warehouse principles

JSON

PARQUET

ORC

AVRO

Scala

Tools

Maven

Kafka

Apache Nifi

Job description

A leading technology distributor in Warsaw is seeking a skilled Big Data Engineer to join an exciting project for a top banking client. You will build and enhance Big Data solutions using Python and Apache Spark. Ideal candidates have a minimum of 3 years' experience with BigData technologies and advanced SQL knowledge. The position requires working on-site once every two weeks, offering a blend of innovative environments and technical challenges.