Data Engineer — Real-Time Cloud Pipelines

Knauf; Katowice; Hybrid; PLN 180,000 - 240,000; Full time

Benefits offered by this job

Permanent contract

Flexible working hours

Weekly remote days

Discounts on sports and leisure

Parking and public transport access

Welcome days and onboarding

Career development support

Creative office environment

Job summary

Knauf Information Services GmbH is seeking a skilled Data Engineer to design, build, and optimize scalable data pipelines on Azure, Databricks, and Kafka. You will automate ETL/ELT workflows, implement data models and governance, and collaborate with data scientists and engineers to deliver trusted datasets.

The role emphasizes security, compliance, and DevSecOps practices, with opportunities to grow in a federated, Agile environment across data platforms and analytics projects.

Qualifications

• You have 5+ years of experience in data engineering, ETL development or big-data technologies, with hands-on expertise in Databricks, Kafka and GitHub.
• You are experienced in designing and optimising ETL/ELT workflows using tools such as Airflow, Azure Data Factory and cloud platforms including Azure Synapse and Databricks.
• You have strong knowledge of data modelling, including Data Vault 2.0, star schema, Party Model and normalisation strategies; DataSphere experience is an advantage.
• You are proficient in SQL and Python, Scala or Java, with experience in Spark, Hadoop, Kafka, Event Hubs, Infrastructure as Code, data security, governance, lineage and regulatory compliance.
• You work effectively in federated Agile and DevSecOps environments, using observability tools such as Monte Carlo, Datadog or Great Expectations, and can influence architectural decisions while improving pipeline performance, availability and engineering standards.

Responsibilities

• Design, build and optimise scalable data pipelines and cloud-native infrastructure for batch and real-time processing, using Azure, Databricks and Kafka.
• Develop and automate ETL/ELT workflows for efficient data ingestion, transformation, storage and orchestration using tools such as Airflow, Azure Data Factory or Prefect.
• Develop scalable data models and ensure data quality, reliability, consistency and governance across data lakes, warehouses and operational data stores.
• Strengthen data security, compliance and operational resilience by applying RBAC, ABAC, encryption, GDPR requirements, monitoring, troubleshooting and recovery practices.
• Collaborate with data scientists, analysts and federated engineering teams to deliver trusted datasets, adopt new technologies, align architecture with business needs and mentor junior engineers.

Skills

Databricks

Kafka

GitHub

SAP Datasphere

Airflow

Azure Data Factory

Azure Synapse

Prefect

Data modelling

SQL

Python

Scala/Java

Spark

Hadoop

Event Hubs

Infrastructure as Code

Data security & governance

Agile & DevSecOps

Observability tools

Tools

Airflow

Azure Data Factory

Prefect

Monte Carlo

Datadog

Great Expectations

Job description

Knauf Information Services GmbH is seeking a skilled Data Engineer to design, build, and optimize scalable data pipelines on Azure, Databricks, and Kafka. You will automate ETL/ELT workflows, implement data models and governance, and collaborate with data scientists and engineers to deliver trusted datasets.

The role emphasizes security, compliance, and DevSecOps practices, with opportunities to grow in a federated, Agile environment across data platforms and analytics projects.