Senior Data Engineer — Data Pipelines Migration & Security

Aqilea; Stockholms kommun; On-site; SEK 900,000 - 1,300,000; Full time

Job summary

Aqilea in Stockholm is seeking a Senior Data Engineer for an interim assignment to re-architect data pipelines across Airflow, Spark, dbt and BigQuery with encryption, retention and deletion controls. You will drive the migration, implement regression harnesses, and define long-term governance for data lifecycle processes.

You will mentor a junior data engineer and help the team own the migrated pipelines, ensuring reliable, auditable pipelines and clear decision records throughout the

Qualifications

• Solid hands-on experience with Airflow, Spark, dbt, BigQuery and Python.
• Led at least one pipeline migration end-to-end.
• Experience refactoring a live pipeline while proving its output is unchanged.
• Experience introducing engineering processes (testing, review, incident handling) to a team and making them stick.
• Mentoring experience with a concrete outcome.

Responsibilities

• Propose and drive the architecture for user-data handling across Airflow, Spark, dbt, and BigQuery: encryption points, derived tables, backfills, partitions.
• Migrate existing pipelines one at a time without changing their output, behind a regression harness you build.
• Design the processes the team keeps after the migration: pipeline change review, testing standards, incident post-mortems, release checklist.
• Remove duplicated steps and copy-pasted logic as you pass through each pipeline.
• Build verification jobs that prove the data‑lifecycle rules hold end to end.
• Pair with and mentor a data engineer on the team, with the goal that the team owns the migrated pipelines independently.
• Share pipeline maintenance and incident handling; fix root causes.
• Document decisions as short architecture decision records.

Skills

Airflow

Spark

dbt

BigQuery

Python

Job description

Aqilea in Stockholm is seeking a Senior Data Engineer for an interim assignment to re-architect data pipelines across Airflow, Spark, dbt and BigQuery with encryption, retention and deletion controls. You will drive the migration, implement regression harnesses, and define long-term governance for data lifecycle processes.

You will mentor a junior data engineer and help the team own the migrated pipelines, ensuring reliable, auditable pipelines and clear decision records throughout the