Senior Data Engineer

Sigmasoftware2

Poland

Sur place

PLN 180 000 - 280 000

Plein temps

Il y a 6 jours
Soyez parmi les premiers à postuler

Obtenez une réponse de cet employeur

Votre CV et votre lettre de motivation, adaptés exactement à ce que l’employeur recherche.

Passez les filtres ATS

Résumé du poste

Sigmasoftware2 is seeking a senior Data Engineer to design and optimize large-scale data pipelines in a production environment. You will build ingestion pipelines for bid, win, and impression data, harmonize datasets, and own the data serving path with strong schema contracts.

You will work with Spark/PySpark, BigQuery, Airflow or Dagster, and feature store concepts while ensuring data quality and regulatory considerations in a fast-paced setting.

Qualifications

  • 5+ years of experience in Data Engineering and end-to-end data pipelines.
  • Expert-level SQL with window functions and incremental processing patterns.
  • Strong Python skills for production-grade pipelines.
  • Hands-on Spark / PySpark experience and large data workloads.
  • Experience with ETL/ELT design and orchestration tools like Airflow or Dagster.
  • Experience with BigQuery and cloud data warehouse design at scale.
  • Solid data modeling and point-in-time correctness understanding.
  • Comfortable in an English-speaking, client-facing environment.

Responsabilités

  • Write and defend diagnostic SQL queries against large production datasets.
  • Build and maintain ingestion pipelines for bid, win, and impression logs into BigQuery.
  • Harmonize fields across datasets and maintain versioned mappings.
  • Develop point-in-time feature tables and aggregation pipelines.
  • Design and maintain conversion and labeling pipelines with delayed labels.
  • Own data serving write path, schema contracts, and publishing flows.
  • Build experimentation infrastructure including traffic splitting and reporting.
  • Perform large-scale backfills and safe reprocessing after mapping changes.
  • Implement data isolation and safe-aggregation controls for advertiser data.
  • Develop automated data quality validation frameworks.
  • Collaborate with Customer engineers and prepare operational docs.
  • Contribute to architecture discussions and platform scalability improvements.

Connaissances

Data engineering
SQL
Python
Spark / PySpark
ETL / ELT patterns
Data modeling
English communication

Outils

BigQuery
Airflow
Dagster
dbt
Terraform
Kubernetes
GCP Dataflow / Beam
PySpark

Description du poste

Responsibilities
  • Write and defend diagnostic SQL queries against large-scale production datasets
  • Build and maintain ingestion pipelines for bid, win, and impression logs into BigQuery
  • Harmonize fields across independently designed datasets and maintain versioned field mappings
  • Develop point‑in‑time‑correct feature tables and aggregation pipelines
  • Design and maintain conversion and labeling pipelines with delayed label handling
  • Own the data serving write path, schema contracts, publishing flows, and freshness SLOs
  • Build experimentation infrastructure including traffic splitting and reporting pipelines
  • Perform large-scale historical backfills and safe reprocessing after mapping changes
  • Implement data isolation and safe‑aggregation controls for advertiser data protection
  • Develop automated data quality validation frameworks
  • Collaborate closely with Customer engineers and prepare operational documentation
  • Contribute to architecture discussions and platform scalability improvements
Requirements
  • 5+ years of experience in Data Engineering
  • At least 2 years of experience working with production ML or large‑scale analytics pipelines
  • Expert‑level SQL skills including window functions and incremental processing patterns
  • Strong Python skills for production‑grade pipeline development
  • Hands‑on experience with Spark or PySpark
  • Experience designing ETL / ELT pipelines with Airflow, Cloud Composer, Dagster, or similar tools
  • Experience working with cloud data warehouses at scale, preferably BigQuery
  • Strong understanding of data modeling and point‑in‑time correctness
  • Experience working with event‑driven or clickstream datasets at very large scale
  • Experience supporting business‑critical production pipelines
  • Upper‑Intermediate English level or higher
WILL BE A PLUS
  • Experience with GCP services including Dataflow, Pub/Sub, GCS, and Beam
  • Experience building streaming or near‑real‑time ingestion systems
  • Understanding of feature stores, train/serve skew, and label leakage prevention
  • Experience in AdTech or auction‑based environments
  • Experience handling delayed or incomplete labels in ML systems
  • Experience with dbt or similar transformation frameworks
  • Experience delivering solutions into Customer‑owned infrastructure
  • Knowledge of GDPR/CCPA‑related privacy engineering practices
  • Experience with experimentation infrastructure and statistical validation pipelines
  • Experience working in hybrid cloud/on‑prem Linux environments
  • Terraform and Kubernetes experience
  • Experience optimizing warehouse cost and performance
Personal Profile
  • Strong analytical and problem‑solving skills
  • Ownership‑oriented mindset
  • Ability to work independently in a client‑facing environment
  • Strong communication and documentation skills
  • Comfortable working in a fast‑paced engineering environment
  • Collaborative and proactive attitude
Obtenez votre examen gratuit et confidentiel de votre CV.

ou faites glisser et déposez votre fichier ici.

Similar jobs

Postes similaires à comparer

GCP Data Engineer — Build Scalable Cloud Data Pipelines
GCP Data Engineer — Build Scalable Cloud Data Pipelines

Lingaro • Polska

À distance
PLN 150 000 - 230 000
GCP Data Engineer - freelance
GCP Data Engineer - freelance

Lingaro • Polska

À distance
PLN 150 000 - 230 000
GCP Data Architect: Scale Data Infrastructure
GCP Data Architect: Scale Data Infrastructure

Lingaro • Polska

Sur place
PLN 240 000 - 420 000
GCP Data Architect - Freelancer (Part-time)
GCP Data Architect - Freelancer (Part-time)

Lingaro • Polska

Sur place
PLN 180 000 - 300 000
Senior Data Scientist
Senior Data Scientist

Sigma Software • Warszawa

Sur place
PLN 260 000 - 480 000
Senior Data Engineer (Java/Scala/Python, Spark, AWS)
Senior Data Engineer (Java/Scala/Python, Spark, AWS)

EPAM Systems • Polska

Sur place
PLN 180 000 - 300 000
Senior Data Engineer
Senior Data Engineer

Softeta • Kraków

Hybride
PLN 180 000 - 260 000
Hybrid workplace model
Flexible working hours
Training budget for growth
+1
Senior Machine Learning Engineer (AdTech)
Senior Machine Learning Engineer (AdTech)

Sigma Software • Województwo mazowieckie

À distance
PLN 240 000 - 360 000
Data Engineer: GCP, Python & BigQuery Architect
Data Engineer: GCP, Python & BigQuery Architect

Infotree Global Solutions Inc. • Polska

À distance
PLN 120 000 - 180 000
Data Engineer (GCP, Snowflake, dbt)
Data Engineer (GCP, Snowflake, dbt)

Addepto • Pologne

Hybride
PLN 180 000 - 240 000
Remote work
Career paths
Training & conferences
+1
Lieux principaux