Senior Data Engineer

Aqilea; Stockholms kommun; On-site; SEK 900,000 - 1,300,000; Full time

Job summary

Aqilea in Stockholm is seeking a Senior Data Engineer for an interim assignment to re-architect data pipelines across Airflow, Spark, dbt and BigQuery with encryption, retention and deletion controls. You will drive the migration, implement regression harnesses, and define long-term governance for data lifecycle processes.

You will mentor a junior data engineer and help the team own the migrated pipelines, ensuring reliable, auditable pipelines and clear decision records throughout the

Qualifications

• Solid hands-on experience with Airflow, Spark, dbt, BigQuery and Python.
• Led at least one pipeline migration end-to-end.
• Experience refactoring a live pipeline while proving its output is unchanged.
• Experience introducing engineering processes (testing, review, incident handling) to a team and making them stick.
• Mentoring experience with a concrete outcome.

Responsibilities

• Propose and drive the architecture for user-data handling across Airflow, Spark, dbt, and BigQuery: encryption points, derived tables, backfills, partitions.
• Migrate existing pipelines one at a time without changing their output, behind a regression harness you build.
• Design the processes the team keeps after the migration: pipeline change review, testing standards, incident post-mortems, release checklist.
• Remove duplicated steps and copy-pasted logic as you pass through each pipeline.
• Build verification jobs that prove the data‑lifecycle rules hold end to end.
• Pair with and mentor a data engineer on the team, with the goal that the team owns the migrated pipelines independently.
• Share pipeline maintenance and incident handling; fix root causes.
• Document decisions as short architecture decision records.

Skills

Airflow

Spark

dbt

BigQuery

Python

Job description

About The Assignment

We are looking for a Senior Data Engineer for an interim position in our Trust Intelligence & Calling department. Note: We prefer on-site but we're considering remote candidate as well. We are re-architecting our data pipelines to meet stricter requirements on user-data lifecycle: encryption, retention, and deletion across historical and derived data. You lead the technical migration and establish the engineering processes that keep the pipelines reliable after it.

• Role: Senior Data Engineer


• Job Ref Id: JD-1292


• Competence level: 5+ years


• Start date: 2026-11-09


• End date: 2027-04-30


• Extension option: Yes


• Scope: 100%


• Location: Stockholm


• Remote work: On-site; remote considered


• Language: English


• Application deadline: 2026-11-01


• Number of consultants: 1


• Security clearance: Not stated

Responsibilities

• Propose and drive the architecture for user-data handling across Airflow, Spark, dbt, and BigQuery: encryption points, derived tables, backfills, partitions.


• Migrate existing pipelines one at a time without changing their output, behind a regression harness you build.


• Design the processes the team keeps after the migration: pipeline change review, testing standards, incident post-mortems, release checklist.


• Remove duplicated steps and copy-pasted logic as you pass through each pipeline.


• Build verification jobs that prove the data‑lifecycle rules hold end to end.


• Pair with and mentor a data engineer on the team, with the goal that the team owns the migrated pipelines independently.


• Share pipeline maintenance and incident handling; fix root causes.


• Document decisions as short architecture decision records.

Requirements

Mandatory

• Deep hands‑on experience with Airflow, Spark, dbt, BigQuery, and Python.


• At least one pipeline migration you led technically from start to finish.


• Experience refactoring a live pipeline while proving its output is unchanged.


• Experience introducing engineering processes (testing, review, incident handling) to a team and making them stick.


• Mentoring experience with a concrete outcome.

Nice-to-have

• A prior project on encryption at rest, data retention, or deletion compliance (for example GDPR).


• Envelope encryption with a key manager (for example GCP KMS), BigQuery AEAD functions, or Tink.


• BigQuery column‑level security, policy tags, Data Catalog.


• Config Connector (KCC) for managing GCP resources from Kubernetes.


• Events with Kafka.