Big Data Engineer — Spark Pipelines (Python/Scala)

Integral Solutions - Informatica Distributor; Warszawa; Hybrid; PLN 223,200 - 334,800; Full time

Job summary

A leading IT service provider is seeking a skilled BigData Engineer for an exciting project with a major banking client. This opportunity is based in Warsaw and involves building and enhancing Big Data solutions. The ideal candidate has a minimum of 3 years of experience in Python or Scala, expertise in Spark, and familiarity with data warehouse principles. The position requires good engineering practices and the ability to work with various data formats.

Qualifications

• Minimum 3 years of experience in Python or Scala programming.
• Commercial experience with BigData technologies; Spark is a must.
• Understanding of database usage scenarios.

Responsibilities

• Write and maintain data pipelines and applications using Python or Scala.
• Work with Apache Spark to process large amounts of data.
• Combine data from different sources into a single system.

Skills

Python programming

Scala programming

Apache Spark

SQL

Data Warehouse principles

Agile processes

Kafka

Tools

Maven

Jira

Confluence

Job description

A leading IT service provider is seeking a skilled BigData Engineer for an exciting project with a major banking client. This opportunity is based in Warsaw and involves building and enhancing Big Data solutions. The ideal candidate has a minimum of 3 years of experience in Python or Scala, expertise in Spark, and familiarity with data warehouse principles. The position requires good engineering practices and the ability to work with various data formats.