Senior Data Engineer: Real-Time Pipelines & AI Analytics

SD Solutions; Warszawa; On-site; PLN 150,000 - 200,000; Full time

Job summary

SD Solutions is seeking a talented Senior Data Engineer to manage end-to-end data pipelines while working with high-volume advertising data and ensuring data quality. The role requires extensive experience in Java, Scala, and Python, along with strong cloud platform knowledge.

Successful candidates will have a minimum of 6 years in commercial development, proficiency in designing scalable architectures, and a strong understanding of data processing. Knowledge of AWS and SQL/NoSQL databases is essential.

Qualifications

• 6+ years of total commercial development experience.
• 4+ years of commercial development experience with Java, Scala (Spark and Flink), and Python.
• Strong knowledge of SQL and NoSQL databases (Vertica and MongoDB experience is a plus).
• Solid experience with AWS cloud platform (or equivalent).
• Experience designing and implementing scalable data architectures.

Responsibilities

• Own end-to-end data pipelines from ingestion to reporting.
• Design, build, and maintain large-scale data processing systems.
• Implement solutions for publishers management and audience analytics.
• Optimize query performance and ensure data reliability.
• Write high-quality, testable code and improve observability.

Skills

Experience with Java

Experience with Scala (Spark and Flink)

Experience with Python

Strong knowledge of SQL databases

Strong knowledge of NoSQL databases

Experience with AWS

Experience with Hadoop

Experience with Kafka

Experience with Kubernetes

Education

BSc in Computer Science or equivalent

Tools

Git

GitLab

Grafana

InfluxDB

Kibana

Job description

SD Solutions is seeking a talented Senior Data Engineer to manage end-to-end data pipelines while working with high-volume advertising data and ensuring data quality. The role requires extensive experience in Java, Scala, and Python, along with strong cloud platform knowledge.

Successful candidates will have a minimum of 6 years in commercial development, proficiency in designing scalable architectures, and a strong understanding of data processing. Knowledge of AWS and SQL/NoSQL databases is essential.