Data Engineer: Scalable Pipelines & Lakehouse

Yora Sigma; Göteborgs kommun; On-site; SEK 700,000 - 900,000; Full time

Job summary

YORA is seeking a Data Engineer to design and evolve scalable data platforms and pipelines that support analysis, validation, and automated regression testing. You will work with Python, SQL, PySpark, and distributed data processing to transform large data volumes into reliable, accessible information.

You will join a team focused on processing, structuring, and managing data, with a primary emphasis on building robust data infrastructure for large-scale automated validation across batch and

Qualifications

• Experience building scalable data pipelines for large datasets.
• Strong programming skills in Python, SQL, and PySpark.
• Understanding of batch and streaming data processing.
• Knowledge of data lakehouse architectures and data modeling.

Responsibilities

• Design, build, and maintain scalable data pipelines for large and complex datasets.
• Develop data processing solutions using Python, SQL, and PySpark.
• Work with batch and streaming data processing.
• Build and optimize ETL/ELT workflows for data transformation and integration.
• Design data models that support analysis, validation, and comparison of large datasets.
• Work with data lakehouse architectures and table formats such as Iceberg or Delta Lake.
• Ensure data quality, lineage, traceability, and reproducibility.
• Develop data flows that enable data-driven system validation and automated regression testing.
• Collaborate with engineers to improve data processing and usage across systems.

Skills

Python

SQL

PySpark

Distributed data processing

Tools

Iceberg

Delta Lake

Job description

YORA is seeking a Data Engineer to design and evolve scalable data platforms and pipelines that support analysis, validation, and automated regression testing. You will work with Python, SQL, PySpark, and distributed data processing to transform large data volumes into reliable, accessible information.

You will join a team focused on processing, structuring, and managing data, with a primary emphasis on building robust data infrastructure for large-scale automated validation across batch and