Lead Data Engineer - Databricks Platform Modernization

Vaillant Group; Katowice; Hybrid; PLN 180,000 - 260,000; Full time

Benefits offered by this job

Hybrid work model

Health & Wellness programs

Time off & balance

Discount on company products

Training & development

Company events

Job summary

Vaillant Group in Katowice, Poland, invites an experienced Lead Data Engineer to shape and modernize our enterprise data platform on Databricks. You will drive scalability, reliability and cost efficiency while mentoring engineers in a hybrid, fast-evolving environment.

You will lead migration to Spark Declarative Pipelines, implement reusable frameworks, optimize Spark performance, and collaborate with stakeholders to translate business needs into robust data solutions.

Qualifications

• Degree in computer science, data engineering, information systems, or a comparable field.
• Several years of data engineering experience with technical leadership or senior responsibilities.
• Deep knowledge of Databricks and Apache Spark, including Delta Lake and modern data architectures.
• Strong programming in Python and SQL; apply software engineering practices like CI/CD and testing.
• Nice to have: Spark Declarative Pipelines & Declarative Automation Bundles.
• Proficient English communication.

Responsibilities

• Define and drive the technical evolution of our enterprise data platform for scalability and cost efficiency.
• Lead our Databricks modernization journey, migrating workloads toward Spark Declarative Pipelines.
• Design and implement core platform capabilities, review critical code, and set engineering standards.
• Optimize Spark performance, cluster utilization, workload efficiency, and Databricks costs.
• Support engineering teams and stakeholders, translating business requirements into robust designs.
• Drive agile delivery: planning, backlog refinement, estimations and deployments.
• Foster innovation with automation, DevOps practices, and reduction of technical debt.

Skills

Databricks

Apache Spark

Python

SQL

CI/CD

Observability

Education

Degree in computer science / data engineering / information systems

Tools

Databricks

Delta Lake

Spark

Job description

Vaillant Group in Katowice, Poland, invites an experienced Lead Data Engineer to shape and modernize our enterprise data platform on Databricks. You will drive scalability, reliability and cost efficiency while mentoring engineers in a hybrid, fast-evolving environment.

You will lead migration to Spark Declarative Pipelines, implement reusable frameworks, optimize Spark performance, and collaborate with stakeholders to translate business needs into robust data solutions.