Senior Data Engineer: Chemical Pipelines & Observability

PRIORITY; Helsinki; Sur place; EUR 80 000 - 120 000; Plein temps

Résumé du poste

Jobtailor in Helsinki is seeking a Senior Data Engineer to design and maintain scalable data pipelines across ingestion, transformation, and publishing stages. You will implement data quality controls, observability, and monitoring for large-scale datasets, working with streaming and batch processing systems such as Event Hubs or Kafka-compatible platforms.

The role requires deep experience in Spark, Databricks, and Azure Data Factory.

Qualifications

• Strong experience with Spark/Databricks.
• Hands-on expertise in Azure Data Factory.
• Knowledge of streaming systems (Event Hubs / Kafka-compatible).
• Strong SQL and Python skills.
• Experience with data quality frameworks and observability.
• Background in large-scale data engineering environments.
• Subject Matter Expert in: Chemical Data Pipelines at scale, heterogeneous scientific data, data quality & observability.

Responsabilités

• Design and maintain scalable data pipelines (ingestion → transformation → publishing).
• Implement robust data quality frameworks and controls.
• Support operational troubleshooting across large-scale datasets.
• Work with streaming and batch processing systems.
• Build and maintain data observability and monitoring solutions.
• Support datasets at scale.

Connaissances

Spark

Databricks

SQL

Python

Streaming platforms

Data quality

Observability

Data engineering

Chemical data pipelines

Outils

Azure Data Factory

Description du poste

Jobtailor in Helsinki is seeking a Senior Data Engineer to design and maintain scalable data pipelines across ingestion, transformation, and publishing stages. You will implement data quality controls, observability, and monitoring for large-scale datasets, working with streaming and batch processing systems such as Event Hubs or Kafka-compatible platforms.

The role requires deep experience in Spark, Databricks, and Azure Data Factory.