Senior Data Engineer (remote EU only) — Azure Databricks + dbt

Valletta.Software | AI-Care; Finland; Remote; EUR 90,000 - 120,000; Full time

Job summary

Valletta.Software | AI-Care is seeking a Senior Data Engineer for a long-term project with a Danish client in pharma/biotech. Remote work is offered with EU residency mandatory, ensuring compliance with EU regulations.

You will implement batch and API ingestion, enforce schemas, and perform data quality checks. You’ll leverage dbt and Databricks for governance and CI/CD in a secure Dev/Test/Prod setup.

Qualifications

• Hands-on experience with Azure Databricks (Spark/Delta Lake).
• Ability to enforce schemas and manage schema drift.
• Implement data quality checks at ingestion.
• Batch and API-based data ingestion experience.
• Strong hands-on dbt experience.
• Monitor pipeline health, SLA adherence, and failures.

Responsibilities

• Implement batch and API-based data ingestion.
• Apply schema enforcement and handle schema drift; run data quality checks at ingestion.
• Work hands-on with dbt for pipeline observability and data governance.
• Monitor job failures, data freshness, and pipeline health against SLA targets.
• Set up alerts, logging, and retry strategies for production pipelines.
• Operate in a controlled Dev/Test/Prod environment with traceability.
• Contribute to CI/CD and MLOps pipelines on Databricks.

Skills

Azure Databricks (Spark/Delta Lake)

Schema enforcement

Data quality checks

Batch and API data ingestion

dbt

Monitoring pipelines

CI/CD & MLOps basics

Tools

dbt

Azure Databricks

Power BI

Job description

We are looking for a Senior Data Engineer to join a long-term project with a Danish client in the pharmaceutical/biotech domain.

Location: Remote, but EU residency is mandatory (we cannot consider candidates outside the European Union).

What you'll do:

• Implement batch and API-based data ingestion.
• Apply schema enforcement and handle schema drift; run data quality checks at ingestion.
• Work hands-on with dbt for pipeline observability and data governance.
• Monitor job failures, data freshness, and pipeline health against SLA targets.
• Set up alerts, logging, and retry strategies for production pipelines.
• Work within a controlled Dev/Test/Prod environment: access control, audit trails, pipeline documentation, traceability to requirements.
• Contribute to CI/CD and MLOps/AI pipelines on Databricks.

Must-haves:

• Strong hands-on experience with Azure Databricks: Spark, Delta Lake.
• Schema enforcement and schema-drift handling.
• Data quality checks at ingestion.
• Batch and API-based data ingestion.
• Strong hands-on experience with dbt.
• Monitoring job failures, data freshness, and pipeline health.
• Alerts, logging, and retry strategies.
• Access control models, audit trails, lineage, traceability.
• Data contracts, pipeline documentation, controlled environments (Dev/Test/Prod).
• EU residency.

Nice-to-have:

• CI/CD & DevOps for Data (Git-based development, CI/CD pipelines for data workflows).
• AI/MLOps — MLOps/AI pipelines on Databricks.
• Exposure to regulatory domains (GxP, clinical trials, pharmacovigilance).
• Dimensional modeling: star schema, OBT, curated marts.
• Tabular Editor, Power BI.
• Pharma/biotech industry experience.

This is a senior, independent role: the ideal candidate is comfortable taking ownership with minimal oversight and has a flexible, self-driven mindset.

A Global, Thriving Team

Join remotely 100+ specialists from 20+ countries, united by a passion for outstanding IT solutions.

Diverse projects:

Fintech, MedTech, AI/ML, e-commerce, and more. Switch teams or industries to broaden your skills.

Support at Every Step Client interview prep: We train you to succeed + give actionable feedback.

Strategic stability:

Well-structured processes, strong management, and long- term vision.

Core values:

Honesty, flexibility, innovation, and a people-first approach.