Senior Data Engineer - Databricks Lakehouse Migration

Sigma Software; Województwo mazowieckie; Hybrid; PLN 240,000 - 360,000; Full time

Job summary

Sigma Software is seeking a Senior Data Engineer to join its Data Engineering Center of Excellence in Warsaw/Województwo mazowieckie. You will lead migration from BigQuery to Databricks, design scalable Lakehouse architectures, and optimize large-scale data pipelines for a retail analytics platform.

Ideal candidates have 5+ years of data engineering, strong Python/SQL, Databricks, Spark, and Delta Lake experience, plus cloud expertise and CI/CD practices.

Qualifications

• 5+ years of professional experience as a Data Engineer.
• Strong programming skills in Python and advanced SQL.
• Hands-on commercial experience with Databricks.
• Strong knowledge of Apache Spark, primarily PySpark.
• Experience designing and building modern cloud-based data platforms.
• Experience developing ETL / ELT pipelines and large-scale data processing solutions.
• Hands-on experience with Delta Lake.
• Experience with Spark Declarative Pipelines.
• Experience with cluster monitoring, metrics analysis, and performance optimization.
• Strong understanding of distributed data processing architectures.
• Solid understanding of data warehousing concepts and dimensional modeling.
• Experience with Airflow or similar orchestration tools.
• Experience optimizing complex analytical SQL workloads.
• Experience implementing CI / CD practices for data engineering platforms.
• Strong troubleshooting and performance optimization skills.
• Ability to work collaboratively in cross-functional international teams.
• Upper-Intermediate or higher English level.

Responsibilities

• Participate in migration of large-scale analytical platform from BigQuery to Databricks.
• Design and implement scalable Lakehouse architectures using Databricks and Delta Lake.
• Analyze ETL/ELT workloads and define migration approaches.
• Develop and optimize data pipelines for large retail datasets and analytics tasks.
• Implement incremental processing strategies and scalable transformation frameworks.
• Build and maintain Spark-based data processing solutions with PySpark.
• Design and maintain medallion architecture layers (Bronze, Silver, Gold).
• Implement data governance and security best practices using Unity Catalog.
• Collaborate with Data Science, Analytics, Product, and Customer Engineering teams.
• Participate in architecture discussions and technical solution design.
• Develop reusable data platform components and engineering standards.
• Conduct code reviews and contribute to platform reliability and maintainability.
• Troubleshoot and optimize complex SQL and Spark workloads.
• Support production deployments and platform modernization activities.

Skills

Python

SQL

Cross-functional collaboration

English proficient

Tools

Databricks

Apache Spark

PySpark

Delta Lake

Airflow

CI/CD

Job description

Sigma Software is seeking a Senior Data Engineer to join its Data Engineering Center of Excellence in Warsaw/Województwo mazowieckie. You will lead migration from BigQuery to Databricks, design scalable Lakehouse architectures, and optimize large-scale data pipelines for a retail analytics platform.

Ideal candidates have 5+ years of data engineering, strong Python/SQL, Databricks, Spark, and Delta Lake experience, plus cloud expertise and CI/CD practices.