Senior Data Engineer: Chemical Pipelines & Observability

Jobtailor; Helsinki; On-site; EUR 80,000 - 120,000; Full time

Job summary

Jobtailor in Helsinki is seeking a Senior Data Engineer to design and maintain scalable data pipelines across ingestion, transformation, and publishing stages. You will implement data quality controls, observability, and monitoring for large-scale datasets, working with streaming and batch processing systems such as Event Hubs or Kafka-compatible platforms.

The role requires deep experience in Spark, Databricks, and Azure Data Factory.

Qualifications

• Strong experience with Spark/Databricks.
• Hands-on expertise in Azure Data Factory.
• Knowledge of streaming systems (Event Hubs / Kafka-compatible).
• Strong SQL and Python skills.
• Experience with data quality frameworks and observability.
• Background in large-scale data engineering environments.
• Subject Matter Expert in: Chemical Data Pipelines at scale, heterogeneous scientific data, data quality & observability.

Responsibilities

• Design and maintain scalable data pipelines (ingestion → transformation → publishing).
• Implement robust data quality frameworks and controls.
• Support operational troubleshooting across large-scale datasets.
• Work with streaming and batch processing systems.
• Build and maintain data observability and monitoring solutions.
• Support datasets at scale.

Skills

Spark

Databricks

SQL

Python

Streaming platforms

Data quality

Observability

Data engineering

Chemical data pipelines

Tools

Azure Data Factory

Job description

Jobtailor in Helsinki is seeking a Senior Data Engineer to design and maintain scalable data pipelines across ingestion, transformation, and publishing stages. You will implement data quality controls, observability, and monitoring for large-scale datasets, working with streaming and batch processing systems such as Event Hubs or Kafka-compatible platforms.

The role requires deep experience in Spark, Databricks, and Azure Data Factory.