Data Engineer (Azure Databricks) in the EU

Andersen Lab; Osterreich; Hybrid; PEN 270,000 - 424,000; Full time

Benefits offered by this job

Remote or hybrid work option

Mentoring program

Annual bonus up to 1000 EUR/month

Private health insurance

Sports compensation

Professional training portal

Referral program

Job summary

Andersen Lab is seeking a Data Engineer (Azure Databricks) to enhance a cloud-based data platform and deliver scalable data solutions for drug development. You will work on data ingestion, governance, and MLOps within a controlled Dev/Test/Prod environment, applying dbt and Spark Delta Lake techniques.

The role requires 4+ years in data engineering, strong SQL skills, and English at an upper-intermediate level, with flexible remote or hybrid work options across EU locations.

Qualifications

• Commercial experience in Data Engineering for 4+ years.
• Strong hands-on experience with Azure Databricks, including Spark and Delta Lake.
• Experience building and maintaining multi-layer data pipelines following the Landing → Raw → Enriched → Curated architecture.
• Experience with schema enforcement and schema-drift handling.
• Ability to implement data quality checks at ingestion.
• Experience orchestrating pipelines using Databricks Jobs / Workflows, including dependencies, retries, alerts, and failure handling.
• Experience with both batch and API-based data ingestion.
• Strong hands-on experience with dbt.
• Ability to monitor job failures, data freshness, and overall pipeline health.
• Experience delivering data in line with defined SLAs.
• Experience implementing alerting, logging, and retry strategies.
• Experience designing and maintaining access control models.
• Understanding of audit trails, data lineage, and end-to-end traceability.
• Experience with logging and evidence generation for audit and compliance purposes.
• Understanding of segregation of duties.
• Experience defining and maintaining data contracts.
• Ability to create and maintain clear pipeline documentation.
• Experience working across controlled Dev, Test, and Prod environments.
• Ability to ensure traceability between requirements and implemented data solutions.
• Level of English – from Upper-Intermediate and above.

Responsibilities

• Implement batch and API-based data ingestion from multiple source systems.
• Apply schema enforcement and handle schema drift; run data quality checks at ingestion.
• Work hands-on with dbt for pipeline observability and data governance tasks.
• Monitor job failures, data freshness, and pipeline health against SLA-based delivery targets.
• Set up alerts, logging, and retry strategies for production pipelines.
• Work within a controlled Dev/Test/Prod environment: access control, audit trails, pipeline documentation, traceability to requirements.
• Contribute to CI/CD and MLOps/AI pipelines on Databricks where the project needs it.

Skills

Data engineering

Azure Databricks

Spark

Delta Lake

dbt

Data quality checks

Data governance

CI/CD for data

Monitoring & alerts

Data contracts

SQL

English (Upper-Intermediate)

Tools

Databricks Jobs/Workflows

APIs ingestion

CI/CD pipelines for data

MLOps on Databricks

Power BI (optional)

Job description

Data Engineer (Azure Databricks) in the EU

This role is not available in your region. View other available roles .

Summary

Andersen is hiring a Data Engineer (Azure Databricks) in the EU for a project enhancing a cloud-based data platform and delivering scalable data solutions for drug development.

The customer is an international company operating in the life sciences sector. It focuses on developing and commercializing specialized products and solutions, supported by research, technology, and partnerships across multiple markets.

The project is focused on developing and enhancing data products for scientific and business stakeholders within the drug development domain. It aims to strengthen the cloud-based data platform and delivery capabilities, enabling scalable data processing, reliable analytics, and effective use of data across the organization.

Responsibilities

• Implementing batch and API-based data ingestion from multiple source systems.
• Applying schema enforcement and handle schema drift; run data quality checks at ingestion.
• Working hands-on with dbt for pipeline observability and data governance tasks.
• Monitoring job failures, data freshness, and pipeline health against SLA-based delivery targets.
• Setting up alerts, logging, and retry strategies for production pipelines.
• Working within a controlled Dev/Test/Prod environment: access control, audit trails, pipeline documentation, traceability to requirements.
• Contributing to CI/CD and MLOps/AI pipelines on Databricks where the project needs it.

Requirements

• Commercial experience in Data Engineering for 4+ years.
• Strong hands-on experience with Azure Databricks, including Spark and Delta Lake.
• Experience building and maintaining multi-layer data pipelines following the Landing → Raw → Enriched → Curated architecture.
• Experience with schema enforcement and schema-drift handling.
• Ability to implement data quality checks at ingestion.
• Experience orchestrating pipelines using Databricks Jobs / Workflows, including dependencies, retries, alerts, and failure handling.
• Experience with both batch and API-based data ingestion.
• Strong hands-on experience with dbt.
• Ability to monitor job failures, data freshness, and overall pipeline health.
• Experience delivering data in line with defined SLAs.
• Experience implementing alerting, logging, and retry strategies.
• Experience designing and maintaining access control models.
• Understanding of audit trails, data lineage, and end-to-end traceability.
• Experience with logging and evidence generation for audit and compliance purposes.
• Understanding of segregation of duties.
• Experience defining and maintaining data contracts.
• Ability to create and maintain clear pipeline documentation.
• Experience working across controlled Dev, Test, and Prod environments.
• Ability to ensure traceability between requirements and implemented data solutions.
• Level of English – from Upper-Intermediate and above.

Desired skills

• Experience with Databricks.
• Experience with SAP S/4HANA; Microsoft 365; Azure; SharePoint; Enterprise integrations.
• Experience with CI/CD and DevOps practices for data, including Git-based development and branching strategies.
• Experience building and maintaining CI/CD pipelines for data workflows.
• Experience promoting code and data solutions across Dev, Test, and Prod environments.
• Ability to integrate automated testing into deployment pipelines.
• Experience with MLOps and AI pipelines on Databricks.
• Exposure to regulated domains such as GxP, clinical trials, and pharmacovigilance.
• Strong SQL engineering skills, including the development of modular and reusable dbt models, tests, and documentation at Analytics Engineer depth.
• Experience with dimensional data modeling, including star schemas, One Big Table (OBT) approaches, and curated data marts.
• Familiarity with Tabular Editor and metadata-as-code concepts.
• Experience with Power BI dataset design, performance optimization, and maintaining KPI and semantic consistency.
• Experience in the pharmaceutical or biotechnology industry.

Reasons to join us

• Experience in teamwork with leaders in FinTech, Healthcare, Retail, Telecom, and others. Andersen cooperates with such businesses as Samsung, Siemens, Johnson & Johnson, BNP Paribas, Ryanair, Mercedes, TUI, Verivox, Allianz, T-Systems, etc..
• The opportunity to change the project and/or develop expertise in an interesting business domain.
• Job conditions – you can work both fully remotely and from the office or can choose a hybrid variant.
• Guarantee of professional, financial, and career growth! The company has introduced systems of mentoring and adaptation for each new employee.
• The opportunity to earn up to an additional 1,000 EUR per month, depending on the level of expertise, which will be included in the annual bonus, by participating in the company's activities.
• Access to the corporate training portal, where the entire knowledge base of the company is collected and which is constantly updated.
• Referral program.
• Private health insurance and sports compensation, depending on the type of employment.

Locations

Austria, Belgium, Bulgaria, Croatia, Cyprus, Czech Republic, Denmark, Estonia, Finland, France, Germany, Hungary, Italy, Latvia, Lithuania, Luxembourg, Poland, Portugal, Romania, Slovakia, Slovenia, Spain, Sweden, The Netherlands