Lead Data Engineer

SNI; Województwo mazowieckie; On-site; PLN 120,000 - 180,000; Full time

Job summary

SNI is serving as a trusted IT Outsourcing partner in line with the needs of World's most prestigious firms and carried out successful projects worldwide.

Lead data engineering work streams and hands-on implementation across multi-functional squads, designing and deploying scalable batch and streaming pipelines, and driving migrations to cloud-native orchestration tools. Establish governance, lineage, metadata, and access security across the data platform.

Qualifications

• 6–9 years of professional experience in data engineering and enterprise data platform delivery.
• Expert-level hands-on experience with Databricks and Apache Spark.
• Strong knowledge of Data Lakehouse architecture and Delta Lake.

Responsibilities

• Lead data engineering work streams and hands-on implementation across multi-functional squads.
• Design and deploy scalable batch and streaming pipelines and data processing frameworks.
• Drive platform migrations from legacy schedulers to modern, cloud-native workflow orchestration tools.
• Establish data governance policies, lineage tracking, metadata management, and role-based security controls.
• Implement data virtualization and unified integration patterns across disparate, multi-source enterprise systems.
• Lead performance tuning, query optimization, cost management, and operational reliability across the data platform.

Skills

Databricks

Apache Spark

Data Lakehouse

Python

SQL

PySpark

Airflow

Databricks Workflows

Dagster

Data governance

Security frameworks

Delta Lake

Tools

Delta Lake

Job description

SNI is serving as a trusted IT Outsourcing partner in line with the needs of World's most prestigious firms and carried out successful projects worldwide.

Responsibilities:

• Lead data engineering work streams and hands-on technical implementation across multi-functional squads.
• Design and deploy resilient, scalable batch and streaming pipelines and data processing frameworks.
• Drive platform migrations from legacy schedulers to modern, cloud-native workflow orchestration tools.
• Establish data governance policies, lineage tracking, metadata management, and role-based security controls (e.g., Unity Catalog).
• Implement data virtualization and unified integration patterns across disparate, multi-source enterprise systems.
• Lead performance tuning, query optimization, cost management, and operational reliability across the data platform.

Skills:

• 6–9 years of professional experience in data engineering and enterprise data platform delivery.
• Expert-level hands-on experience with Databricks and Apache Spark.
• Deep knowledge of Data Lakehouse architecture (Delta Lake, medallion architecture, schema enforcement).
• Advanced proficiency in Python, SQL, and PySpark for data transformation and distributed computing.
• Strong background in workflow orchestration tools (e.g., Airflow, Databricks Workflows, Dagster) and scheduler migration projects.
• Practical experience implementing data governance, cataloging, metadata, and data access security frameworks.
• Proven track record architecting data virtualization and integration solutions across heterogeneous cloud and on-premise sources.