Senior Data Engineer: Real-Time ML Data Platform

Sigma Software; Warszawa; On-site; PLN 180,000 - 320,000; Full time

Job summary

Sigma Software in Warszawa, Poland, is seeking a Senior Data Engineer to design and optimize large-scale data infrastructure for a real-time AdTech platform. You will build production-grade ML-oriented data systems, implement robust ingestion pipelines, and collaborate with customer engineers to ensure data quality, reliability, and scalability across cloud-native environments.

Key focus areas include writing diagnostic SQL queries, building BigQuery data pipelines for bid/win/impression logs,

Qualifications

• 5+ years of experience in Data Engineering.
• At least 2 years of experience with production ML or large-scale analytics pipelines.
• Expert-level SQL skills including window functions and incremental processing patterns.
• Strong Python skills for production-grade pipeline development.
• Hands-on experience with Spark or PySpark.
• Experience designing ETL / ELT pipelines with Airflow, Cloud Composer, Dagster, or similar tools.
• Experience working with cloud data warehouses at scale, preferably BigQuery.
• Strong understanding of data modeling and point-in-time correctness.
• Experience working with event-driven or clickstream datasets at very large scale.
• Experience supporting business-critical production pipelines.
• Upper-Intermediate English level or higher.

Responsibilities

• Write and defend diagnostic SQL queries against large-scale production datasets.
• Build and maintain ingestion pipelines for bid, win, and impression logs into BigQuery.
• Harmonize fields across independently designed datasets and maintain versioned field mappings.
• Develop point-in-time-correct feature tables and aggregation pipelines.
• Design and maintain conversion and labeling pipelines with delayed label handling.
• Own the data serving write path, schema contracts, publishing flows, and freshness SLOs.
• Build experimentation infrastructure including traffic splitting and reporting pipelines.
• Perform large-scale historical backfills and safe reprocessing after mapping changes.
• Implement data isolation and safe-aggregation controls for advertiser data protection.
• Develop automated data quality validation frameworks.
• Collaborate closely with Customer engineers and prepare operational documentation.
• Contribute to architecture discussions and platform scalability improvements.

Skills

Data Engineering

ML pipelines

SQL

Python

Spark

ETL/ELT

BigQuery

Data modeling

English

Tools

Airflow

Cloud Composer

Dagster

BigQuery

PySpark

Job description

Sigma Software in Warszawa, Poland, is seeking a Senior Data Engineer to design and optimize large-scale data infrastructure for a real-time AdTech platform. You will build production-grade ML-oriented data systems, implement robust ingestion pipelines, and collaborate with customer engineers to ensure data quality, reliability, and scalability across cloud-native environments.

Key focus areas include writing diagnostic SQL queries, building BigQuery data pipelines for bid/win/impression logs,