Data Engineer (Spoke)

IRIUM Portugal; Lisboa; Hybrid; EUR 45,000 - 75,000; Full time

Benefits offered by this job

Flexible pay according to experience

Medical insurance as social benefit

Access to technological training

Job summary

IRIUM Portugal is seeking a Data Engineer (Spoke) to join Lisbon-based hybrid team. You will upload data to the Big Data Platform, prepare data for use cases, and build ready-to-use data pipelines, collaborating with the service provider for complex tasks.

We expect 4 years of experience, English C1, and strong knowledge of Spark, Python, AWS ingestion, and distributed systems. You will work with Docker deployments, CI/CD pipelines, and agile Scrum processes in a dynamic automotive-related data

Qualifications

• 4 years experience in data engineering or equivalent.
• English level C1.
• Proficient in Spark, Python, R, Scala, Java and Kotlin.
• Understanding of AWS Cloud Ingestion solutions and S3/databases.
• Knowledge of distributed systems and data pipelines.
• Experience with Docker-based deployment and resource management.
• Familiarity with Scrum/agile software development.

Responsibilities

• Uploading data to the Big Data Platform.
• Targeted preparation of data for specific use cases and data profiling.
• Development of ready-to-use data pipelines.
• Coordination of the service provider's activities for complex tasks.

Skills

Spark

Python

R

Scala

Java

Kotlin

Docker

CI/CD

Workflow Managers

Agile / Scrum

AWS Cloud Ingestion

Tools

CI/CD tools

Job description

At IRIUM we want you to always chase your dreams. Here, prepare yourself to conquer your goals, while enjoying the journey.

We are currently looking for Data Engineer (Spoke)

Requirements

• 4 years experience
• English C1

Role and Responsibilities

• Uploading data to the Big Data Platform
• Targeted preparation of data for specific use cases and creation of data profiles
• Development of ready-to-use data pipelines
• Coordination of the service provider's activities (only complex)

Requirements And Skills

• Proficient in the programming languages Apache Spark, Python, R, Scala, Java and Kotlin
• Understanding of AWS Cloud Ingestion solutions
• Understanding of efficient data processing and attitude in S3 and databases (on-prem and cloud)
• Knowledge of relevant source systems and their data types and structures
• Knowledge in the implementation of batch and real-time interfaces
• Extended expertise in the field of software engineering for the series production of productive data products (incl. testing, deployment, code quality, logging, monitoring, operation)
• Experience in the development of distributed systems
• Experience in Docker-based deployment and Resource Manager
• Experience in CI/CD pipelines and tools
• Knowledge and practical experience with Workflow Managers
• Detailed knowledge of agile software development according to Scrum
• Technical knowledge of the automotive industry"

Location: Lisbon, hybrid

What do we offer?

• An innovative and growing company, with a lot of opportunities for professional development.
• Retribution according to your experience and performance. Access to flexible pay and medical insurance as a social benefit.
• Unlimited access to technological training in free mode.

IRIUM is a company with dynamic and proactive professionals. Our values are responsibility and commitment to work quality. This is the spirit we are looking for at IRIUM, whatever your age is. If you recognize yourself in this, this is your company!

We can build the future together. Let's talk!

At IRIUM we defend a world without stereotypes or limitations and we believe in equality for all, principles that we subscribe to in our Equality Plan and Code of Ethics, guaranteeing equal treatment and opportunities regardless of any personal, physical or social condition.