Remote Senior Data Engineer - Databricks & dbt (EU)

Valletta.Software | AI-Care; Finland; Remote; EUR 90,000 - 120,000; Full time

Job summary

Valletta.Software | AI-Care is seeking a Senior Data Engineer for a long-term project with a Danish client in pharma/biotech. Remote work is offered with EU residency mandatory, ensuring compliance with EU regulations.

You will implement batch and API ingestion, enforce schemas, and perform data quality checks. You’ll leverage dbt and Databricks for governance and CI/CD in a secure Dev/Test/Prod setup.

Qualifications

• Hands-on experience with Azure Databricks (Spark/Delta Lake).
• Ability to enforce schemas and manage schema drift.
• Implement data quality checks at ingestion.
• Batch and API-based data ingestion experience.
• Strong hands-on dbt experience.
• Monitor pipeline health, SLA adherence, and failures.

Responsibilities

• Implement batch and API-based data ingestion.
• Apply schema enforcement and handle schema drift; run data quality checks at ingestion.
• Work hands-on with dbt for pipeline observability and data governance.
• Monitor job failures, data freshness, and pipeline health against SLA targets.
• Set up alerts, logging, and retry strategies for production pipelines.
• Operate in a controlled Dev/Test/Prod environment with traceability.
• Contribute to CI/CD and MLOps pipelines on Databricks.

Skills

Azure Databricks (Spark/Delta Lake)

Schema enforcement

Data quality checks

Batch and API data ingestion

dbt

Monitoring pipelines

CI/CD & MLOps basics

Tools

dbt

Azure Databricks

Power BI

Job description

Valletta.Software | AI-Care is seeking a Senior Data Engineer for a long-term project with a Danish client in pharma/biotech. Remote work is offered with EU residency mandatory, ensuring compliance with EU regulations.

You will implement batch and API ingestion, enforce schemas, and perform data quality checks. You’ll leverage dbt and Databricks for governance and CI/CD in a secure Dev/Test/Prod setup.