Remote Data Engineer (Spark) — Cloud & Big Data Platform

Addepto; Białystok; Hybrid; PLN 180,000 - 210,000; Full time

Benefits offered by this job

Remote work opportunities

Flexible working hours

Training & conference budget

Medical care and well-being support

Career development

Job summary

Addepto is a leading AI consulting and data engineering company that builds scalable, ROI-focused AI solutions for global enterprises and startups. The Data Engineer (Spark) role offers a chance to work on large automotive data platforms and cutting-edge data technologies.

You will contribute to streaming and batch data processing, leverage Spark, Cloudera, Airflow, Iceberg, Python, and AWS, and collaborate with cross-functional teams in a dynamic, supportive environment.

Qualifications

• At least 3 years of commercial experience implementing, developing, or maintaining Big Data systems.
• Strong programming skills in Python (or Java/Scala): writing clean, OOP code.
• Hands-on with Spark, Cloudera, Kafka, Data Platform, Airflow, NiFi, Docker, and Iceberg.
• Excellent understanding of dimensional data and data modeling techniques.
• Experience deploying solutions in cloud environments.
• Consulting experience with client management and effective communication.
• Fluent English (C1 level).
• Bachelor’s degree in technical or mathematical studies.

Responsibilities

• Develop and maintain a high-performance data processing platform for automotive data, ensuring scalability and reliability.
• Design and implement data pipelines that process large volumes of data in both streaming and batch modes.
• Optimize data workflows using Spark, Cloudera, and Airflow.
• Work with data lake technologies (e.g., Iceberg) to manage structured and unstructured data.
• Collaborate with cross-functional teams to understand data requirements and integration needs.
• Monitor and troubleshoot the platform to ensure high availability, performance, and accuracy.
• Leverage AWS for infrastructure management and scaling.
• Write and maintain high-quality Python (or Java/Scala) code for data processing tasks and automation.

Skills

Python

SQL

Spark

Airflow

Cloudera

Education

Bachelor's degree in technical or mathematical studies

Tools

Docker

Databricks

Kubernetes

Kafka

NiFi

Trino

Hudi

Java

Scala

Iceberg

AWS

Job description

Addepto is a leading AI consulting and data engineering company that builds scalable, ROI-focused AI solutions for global enterprises and startups. The Data Engineer (Spark) role offers a chance to work on large automotive data platforms and cutting-edge data technologies.

You will contribute to streaming and batch data processing, leverage Spark, Cloudera, Airflow, Iceberg, Python, and AWS, and collaborate with cross-functional teams in a dynamic, supportive environment.