Data Engineer: Databricks, PySpark & Delta Lake

Sigma Software; Warszawa; On-site; PLN 180,000 - 280,000; Full time

Job summary

Sigma Software is seeking a Middle Data Engineer to build scalable data solutions for a leading AdTech and analytics customer. You will work with modern Data Lakehouse architecture, Databricks, PySpark, Delta Lake, and cloud-native platforms to process large volumes of data in batch and near real-time environments.

You will collaborate with engineers, analysts, and stakeholders to implement robust ETL, ensure data quality, and stay current with evolving data engineering technologies.

Qualifications

• 2+ years of experience in Big Data development and database design.
• Strong hands-on experience with SQL.
• Hands-on experience with PySpark.
• Experience developing and maintaining data pipelines in Databricks.
• Experience with Delta Lake.
• Experience with Medallion Architecture (Bronze, Silver, Gold layers).
• Strong analytical and problem-solving skills.
• Intermediate or higher English level.

Responsibilities

• Develop and maintain data pipelines for processing large volumes of data in batch and near real-time environments.
• Work with Databricks, PySpark, and SQL to transform, validate, and optimize data.
• Support implementation of Data Lakehouse solutions following engineering standards and best practices.
• Build, test, and automate ETL processes and data workflows.
• Improve data quality, organization, and reliability across platforms.
• Collaborate with engineers, analysts, and stakeholders to deliver scalable solutions.
• Troubleshoot, optimize, and monitor data pipelines.
• Stay up to date with modern data engineering technologies.
• Ensure deliverables meet quality standards and project requirements.

Skills

Big Data development

SQL

PySpark

Databricks

Delta Lake

Medallion Architecture

Analytical thinking

English language

Tools

AWS S3

Databricks

Job description

Sigma Software is seeking a Middle Data Engineer to build scalable data solutions for a leading AdTech and analytics customer. You will work with modern Data Lakehouse architecture, Databricks, PySpark, Delta Lake, and cloud-native platforms to process large volumes of data in batch and near real-time environments.

You will collaborate with engineers, analysts, and stakeholders to implement robust ETL, ensure data quality, and stay current with evolving data engineering technologies.