Data engineer (python) in Feature Store

Websensa; Warszawa; Híbrido; PLN 110 000 - 193 000; Tempo integral

Vantagens oferecidas por esta oferta de emprego

Hybrid work

B2B cooperation

Growth opportunities

Resumo da oferta

Websensa ищет инженера по данным уровня Mid/ Mid+ для разработки платформы Feature Store. Ваша задача — проектировать и развивать production data pipeline, обрабатывать большие объемы данных и подготавливать признаки для ML-моделей.

Требуется сильное владение Python и опыт работы с PySpark, SQL, Airflow и Docker. Гибридная работа по польскому рынку, ставка 80–140 PLN/ч netto + VAT (B2B). Участие в проектах AdTech и e-commerce.

Qualificações

• Очень хорошие знания Python.
• Опыт создания и поддержки production data pipelines.
• Практические знания PySpark или аналогичного распределенного инструмента.
• Очень хорошие знания SQL.
• Навыки работы с инструментами оркестрации данных: Airflow или Argo Workflows.
• Практические знания Git и Docker.
• Знание принципов разработки надежных конвейеров данных.
• Умение писать читаемый, тестируемый код.
• Готовность общаться на польском языке.

Responsabilidades

• Разработать и поддерживать платформу Feature Store.
• Строить конвейеры обработки данных на Python.
• Обрабатывать большие объемы данных с использованием PySpark/Polars.
• Интегрировать данные с хранилищами и аналитическими системами.
• Оркестрировать и автоматизировать потоки данных.
• Разрабатывать компоненты и сервисы поддержки платформы.
• Подготавливать и сервировать признаки для ML-моделей, в онлайн-сценариях.
• Оптимизировать производительность, стоимость и масштабируемость процессов.

Conhecimentos

Python

SQL

Polish language

Ferramentas

PySpark

Airflow

Argo Workflows

Git

Docker

Polars

ClickHouse

Delta Lake

Apache Iceberg

Descrição da oferta de emprego

Описание: We are looking for a Mid / Mid+ Data Engineer to join the team developing a Feature Store platform- a solution responsible for computing, storing, and serving features used by machine learning models. You will work on developing a production data platform, building and optimizing pipelines that process large volumes of information. Projects include retrieving data from analytical warehouses, distributed processing, preparing and storing data, and serving it to systems and ML models. Python is the main programming language, and our daily technologies include PySpark, Polars, Airflow, ClickHouse, Delta Lake, and Apache Iceberg. We also use Go in selected components. This role is for someone with strong Python skills and experience processing large datasets who wants to develop practical, prod

Задачи

• Design, develop, and maintain the Feature Store platform.
• Build and develop data processing pipelines in Python.
• Process large volumes of data using tools such as PySpark and Polars.
• Integrate with data warehouses and analytical systems.
• Prepare and store data in formats used in data lake and lakehouse architectures.
• Orchestrate and automate data flows.
• Develop components and services that support the data platform.
• Prepare and serve features used by ML models, including in online scenarios.
• Optimize the performance, cost, and scalability of data processing workflows.
• Ensure data quality, correctness, and consistency, including validation, schema handling, and process idempotency.
• Debug, analyze errors, and solve technical problems.
• Work with containerization and runtime environments based on Docker and Kubernetes.
• Ensure code quality, testability, and maintainability of solutions.
• Participate in designing data architecture and technical solutions.
• Collaborate with data scientists, ML engineers, backend developers, architects, and product owners.
• Work with platform users, help solve problems, and advise on how to use the available solutions.
• Participate in task estimation, code reviews, and project planning.
• Use AI tools that support programming and automate developers' work.

Требования

• Very good knowledge of Python.
• Experience building, developing, and maintaining production data pipelines.
• Practical knowledge of PySpark or another distributed processing tool.
• Very good knowledge of SQL.
• Experience processing large datasets, including volumes exceeding available memory.
• Knowledge of data flow orchestration tools, e.g. Airflow or Argo Workflows.
• Practical knowledge of Git and Docker.
• Knowledge of good practices for designing and developing data pipelines.
• Ability to write readable, testable, and maintainable code.
• Ability to debug and analyze technical problems.
• Independence in completing tasks and accountability for delivered solutions.
• A problem-solving mindset and a focus on finding practical solutions.
• Attention to code quality, data correctness, and good development practices.
• Openness to communicating with platform users, understanding their needs, and supporting them in using the solutions being developed.
• Willingness to continuously develop and learn new technologies.
• Ability to communicate in Polish.
• Readiness to work in a hybrid model.

Будет плюсом

• Experience with lakehouse formats and technologies, e.g. Delta Lake or Apache Iceberg
• Knowledge of the Polars library
• Experience with analytical databases, e.g. ClickHouse or StarRocks
• Knowledge of Redis or other solutions used for fast data serving
• Knowledge of Go
• Experience with the Django framework or building backend services that support data platforms
• Experience with Feature Store platforms, e.g. Feast, Tecton, or Hopsworks
• Knowledge of MLOps and preparing data for machine learning models
• Knowledge of tools for distributed data processing and managing computational tasks, e.g. Ray or Kubeflow
• Experience with message queue systems and data streaming, e.g. Kafka
• Experience with Kubernetes and cloud environments, e.g. GCP or Azure
• Experience with CI/CD and automating development processes
• Experience writing unit, integration, and data quality tests
• Ability to work with monitoring, logging, and observability tools
• Ability to work effectively with AI tools that support programming, e.g. Claude Code, Cursor, Codex, Gemini CLI, GitHub Copilot, or similar tools
• Openness to an agentic coding approach and automating some development tasks

Условия

• B2B cooperation.
• Hybrid work.
• Full-time or cooperation at a similar number of hours.
• Commercial, production, and implementation projects.
• Working language: Polish.
• Proposed rate: PLN 80-140 net per hour + VAT on B2B. The final rate depends on experience, independence, knowledge of Python and big data technologies, experience building production data pipelines, and readiness to work with AI tools that support programming.
• Work on developing a production Feature Store platform.
• Participation in AdTech and e-commerce projects at the intersection of data engineering and machine learning.
• Work with large volumes of data and big data technologies.
• Opportunity to develop existing systems and build new components from scratch.
• Cooperation with an experienced technology and data science team.
• Opportunity to influence data architecture, the choice of technical solutions, and code quality.
• Work with a modern data processing ecosystem, including Spark, Polars, Delta Lake, Iceberg, and ClickHouse.
• Room to propose technical improvements and optimize existing solutions.
• Opportunity to use modern AI tools that support programming.
• Opportunity to develop towards a Senior Data Engineer, ML Platform Engineer, Tech Lead, or Data Architect role.