data engineer for clickstream data

Allegro; Warszawa; On-site; PLN 163,000 - 232,000; Full time

Benefits offered by this job

Medical package

Sports program

Lunch vouchers

English lessons

Training budget

Volunteering day

Job summary

HireHi ищет Data Engineer в Варшаве для разработки и масштабирования пайплайнов в Vento-сервисе, который стандартизирует и валидирует кликстрим-события Allegro.

Вы будете работать с Python, Spark, Kubernetes, Docker; гибридный график 4/1, 30 дней удаленной работы, конкурентное вознаграждение и возможность обучения за счет компании.

Qualifications

• Опыт построения стабильных конвейеров обработки больших данных.
• Знание Python и Spark; возможна работа с Java backend.
• Понимание принципов качества данных и валидации контрактов событий.
• Готовность к code-review и развитию навыков; автономность.
• Умение сообщать о тех. проблемах и активно взаимодействовать в команде.
• Интерес к связке Big Data, Backend и Frontend.
• Желание использовать AI-инструменты для автоматизации задач.
• Русский C1 и английский B2.
• Плюсом: Scala.

Responsibilities

• Разрабатывать и поддерживать пайплайны для высокочастотных клик-стримов.
• Внедрять детекцию аномалий до загрузки в хранилище.
• Согласовывать требования данных с участниками проекта.
• Участвовать в обсуждении архитектуры и ревью кода.
• Обеспечивать автономность и ответственность за задачи.

Skills

Big Data

Python

Spark

Java backend

Data quality

Anomaly detection

Streaming data

Autonomy

Communication

English (B2)

Tools

Kubernetes

Docker

Consul

GitHub

GitHub Actions

Job description

Описание:

Vento is a service that standardizes and validates Allegro clickstream events. It supports high-quality data for AI agents and core business dashboards.

Задачи:

• Design, scale, develop, and maintain data validation pipelines for high-volume clickstream data within the Vento service
• Implement anomaly detection mechanisms to ensure data integrity and quality before ingestion into the core Data Assets layer
• Collaborate with cross-functional stakeholders to reconcile data requirements
• Participate in engineering discussions, code reviews, and cross-team initiatives

Требования:

• Production-proven experience in Big Data ecosystems and building stable data pipelines
• Proficiency in Python and Spark, with openness to working with Java backend applications
• Understanding of data quality principles, event contract validation, anomaly detection, and streaming data processing
• Openness to feedback, participation in code reviews, and willingness to develop skills
• Proactive communication, ability to signal technical problems, and ownership of tasks with a high level of autonomy
• Broad technical perspective and interest in how Big Data, Backend, and Frontend interconnect
• Enthusiasm for using AI coding tools to automate repetitive tasks and optimize workflows
• Polish at C1 level and English at B2 level
• Будет плюсом: Scala

Условия:

• Flexible working hours; working hours start between 7:00 a.m. and 10:00 a.m
• Hybrid schedule: 4/1, with 30 days of occasional remote work
• Data Engineer salary: 14 600 - 20 825 PLN under an employment contract, with tax-deductible cost
• Annual bonus based on annual performance and company results
• Well-located offices with fully equipped kitchens, bicycle parking, and green terraces
• Work tools include a 16\" or 14\" MacBook Pro or corresponding Dell with Windows, plus necessary accessories
• Cafeteria benefits include medical, sports, or lunch packages, insurance, and purchase vouchers
• Paid English classes related to the specific nature of the job
• Training budget, inter-team tourism, hackathons, and an internal learning platform with multiple trainings
• Additional day off for volunteering
• Social events for Allegro employees
• Access to modern AI tools and an internal technology ecosystem including Kubernetes, Docker, Consul, GitHub, and GitHub Actions
• The role offers autonomy in team organization, continuous development, and opportunities to try new things
• Solutions are used by millions of customers
• Allegro Scale includes over 1000 microservices, an open-source data bus (Hermes) with 300K+ rps, a Service Mesh with 1M+ rps, tens of petabytes of data, and production-used machine learning
• Allegro Tech participates in industry conferences and tech communities, runs a blog and podcasts, leads guilds, and organizes the Allegro Tech Meeting