Hybrid Data Engineer - Big Data Pipelines (Spark/AWS)

IRIUM Portugal; Lisboa; Hybrid; EUR 45,000 - 75,000; Full time

Benefits offered by this job

Flexible pay according to experience

Medical insurance as social benefit

Access to technological training

Job summary

IRIUM Portugal is seeking a Data Engineer (Spoke) to join Lisbon-based hybrid team. You will upload data to the Big Data Platform, prepare data for use cases, and build ready-to-use data pipelines, collaborating with the service provider for complex tasks.

We expect 4 years of experience, English C1, and strong knowledge of Spark, Python, AWS ingestion, and distributed systems. You will work with Docker deployments, CI/CD pipelines, and agile Scrum processes in a dynamic automotive-related data

Qualifications

• 4 years experience in data engineering or equivalent.
• English level C1.
• Proficient in Spark, Python, R, Scala, Java and Kotlin.
• Understanding of AWS Cloud Ingestion solutions and S3/databases.
• Knowledge of distributed systems and data pipelines.
• Experience with Docker-based deployment and resource management.
• Familiarity with Scrum/agile software development.

Responsibilities

• Uploading data to the Big Data Platform.
• Targeted preparation of data for specific use cases and data profiling.
• Development of ready-to-use data pipelines.
• Coordination of the service provider's activities for complex tasks.

Skills

Spark

Python

R

Scala

Java

Kotlin

Docker

CI/CD

Workflow Managers

Agile / Scrum

AWS Cloud Ingestion

Tools

CI/CD tools

Job description

IRIUM Portugal is seeking a Data Engineer (Spoke) to join Lisbon-based hybrid team. You will upload data to the Big Data Platform, prepare data for use cases, and build ready-to-use data pipelines, collaborating with the service provider for complex tasks.

We expect 4 years of experience, English C1, and strong knowledge of Spark, Python, AWS ingestion, and distributed systems. You will work with Docker deployments, CI/CD pipelines, and agile Scrum processes in a dynamic automotive-related data