Senior Data Engineer (Stockholm)

Poland and Central-Eastern Europe; United States; À distance; USD 134 000 - 201 000; Plein temps

Résumé du poste

Xebia is seeking a senior data engineering lead to architect and migrate complex data pipelines using Airflow, Spark, dbt and BigQuery. You will guide code reviews, testing standards, and incident handling, while mentoring a data engineer to own pipelines end-to-end.

Remote role open to EU-region professionals with an EU work permit. You will collaborate with cross-functional teams to ensure outputs remain unchanged during migration and enforce robust data governance practices.

Qualifications

• Deep hands-on experience with Airflow, Spark, dbt, BigQuery and Python.
• Track record of leading at least one pipeline migration from start to finish.
• Experience refactoring a live pipeline and proving that outputs remained unchanged.
• Proven ability to introduce testing, code review and incident handling and help a team adopt them.
• Mentoring experience with concrete outcomes.

Responsabilités

• Proposing and driving the architecture for user-data handling across Airflow, Spark, dbt and BigQuery, including encryption points, derived tables, backfills and partitioning.
• Migrating existing pipelines one at a time while keeping their outputs unchanged, using a regression harness you build.
• Designing and introducing processes the team will use after the migration, including pipeline change reviews, testing standards, incident post-mortems and release checklists.
• Refactoring pipelines to remove duplicated steps and copy-pasted logic.
• Building verification jobs to demonstrate that data-lifecycle rules are followed end to end.
• Pairing with and mentoring a data engineer, with the goal of enabling the team to own the migrated pipelines independently.
• Sharing pipeline maintenance and incident handling, and addressing root causes.
• Documenting key decisions in concise architecture decision records.

Connaissances

Python

Data engineering

Mentoring

Technical leadership

Outils

Airflow

Spark

dbt

BigQuery

Description du poste

Xebia is a global AI-first, digital transformation, and engineering partner.With over 25 years of experience and a team of 5,000 professionals across 16 countries, we help organizations design and build scalable products, platforms, and data-driven solutions.

We specialize in Artificial Intelligence, Data and Cloud, Intelligent Automation, and Digital Products, combining deep technical expertise with a strong focus on engineering excellence and a people-first culture.

In the CEE region, we’re a team of nearly 1,000 experts delivering modern applications, data platforms, and AI solutions for clients such as Millennium, ING, Play, Edenred, Arabian Drilling, FedEx, Leroy Merlin, Truecaller, Volotea, Schmitz Cargobull, and many, many more. We work with leading technologies including AWS, Azure, GCP, Databricks, and Snowflake, and combine strong engineering culture with a consulting mindset and a continuous focus on growth and knowledge sharing.

You will be:

• Proposing and driving the architecture for user-data handling across Airflow, Spark, dbt and BigQuery, including encryption points, derived tables, backfills and partitioning.

• Migrating existing pipelines one at a time while keeping their outputs unchanged, using a regression harness you build.

• Designing and introducing processes the team will use after the migration, including pipeline change reviews, testing standards, incident post-mortems and release checklists.

• Refactoring pipelines to remove duplicated steps and copy-pasted logic.

• Building verification jobs to demonstrate that data-lifecycle rules are followed end to end.

• Pairing with and mentoring a data engineer, with the goal of enabling the team to own the migrated pipelines independently.

• Sharing pipeline maintenance and incident handling, and addressing root causes.

• Documenting key decisions in concise architecture decision records.

Your profile:

• Deep hands-on experience with Airflow, Spark, dbt, BigQuery and Python.

• A track record of technically leading at least one pipeline migration from start to finish.

• Experience refactoring a live pipeline and proving that its output remained unchanged.

• Proven ability to introduce engineering practices such as testing, code review and incident handling, and help a team adopt them consistently.

• Mentoring experience with a concrete outcome.

• A hands-on approach to technical decisions, combined with the ability to support and guide other engineers.

Work from the European Union region and a work permit are required.

Nice to have:

• Experience with encryption at rest, data retention or deletion compliance, for example GDPR.

• Experience with envelope encryption and a key manager such as GCP KMS, BigQuery AEAD functions or Tink.

• Experience with BigQuery column-level security, policy tags or Data Catalog.

• Experience with Config Connector (KCC) for managing GCP resources from Kubernetes.

• Experience working with Kafka events.

Recruitment Process:

CV review – HR call – Interview – Client Interview – Decision