Senior Big Data Engineer - Spark, Scala, Python, AWS

DCG Poland; Łódź, Gdynia; On-site; PLN 167,400 - 279,000; Full time

Benefits offered by this job

Private medical care

Co-financing for the sports card

Constant support of dedicated consultant

Job summary

DCG Poland is looking for experienced professionals to develop distributed BigData processing pipelines. Candidates should have 7+ years of expertise in Spark and Scala, along with substantial experience in Python and Robot Framework. You will collaborate with DevOps and QA teams in an Agile environment, maintaining a focus on quality and continuous improvement. The role offers private medical care and support from a dedicated consultant.

Qualifications

• 7+ years of experience in Spark & Scala.
• 5+ years of development experience with Python.
• 3+ years experience with Robot Framework.
• 3+ years experience with CI/CD tools such as Jenkins.
• Strong SQL skills with previous experience in creating data flows.

Responsibilities

• Building distributed and highly parallelized BigData processing pipeline.
• Leverage Spark for data transformation.
• Work closely with DevOps, QA and Product Management teams.

Skills

Spark & Scala

Python

Robot Framework

CI/CD tools (Jenkins)

Hadoop

SQL

BitBucket and GIT

AWS

Linux and bash

Jira

Docker (nice to have)

KAFKA, IBM MQ (nice to have)

Tools

Jenkins or Bamboo

Jira

bitbucket

Job description

DCG Poland is looking for experienced professionals to develop distributed BigData processing pipelines. Candidates should have 7+ years of expertise in Spark and Scala, along with substantial experience in Python and Robot Framework. You will collaborate with DevOps and QA teams in an Agile environment, maintaining a focus on quality and continuous improvement. The role offers private medical care and support from a dedicated consultant.