Data engineer platformy CDP

Finture; Warszawa; On-site; PLN 120,000 - 180,000; Full time

Benefits offered by this job

Финансирование обучения

Международная среда

Возможность карьерного роста

Job summary

Finture ищет Data Engineer для разработки и поддержки CDP на основе Data Lakehouse в Google Cloud Platform. Ваша роль — проектировать и оптимизировать ETL/ELT пайплайны, работать с Bronze/Silver/Gold слоями и обеспечивать качество данных.

Вы будете взаимодействовать с IT-терминами в международной среде, настраивать мониторинг и участвовать в проектах с использованием Datastream, Pub/Sub и Dataform. Требуется опыт с GCP и знание английского языка на рабочем уровне.

Qualifications

• Опыт реализации облачных проектов в роли data engineer.
• Минимум 6 месяцев практики с GCP (BigQuery, Cloud Storage, Dataflow).
• Знание SQL (BigQuery dialect) и Python (Pandas).
• Опыт работы с медальонной архитектурой Bronze/Silver/Gold.
• Знание процессов CI/CD и мониторинга облачных сервисов.

Responsibilities

• Разработка и поддержка CDP на основе архитектуры Data Lakehouse в GCP.
• Проектирование и оптимизация ETL/ELT пайплайнов (batch и streaming).
• Управление трансформациями данных по Bronze/Silver/Gold слоям.
• Оркестрация потоков данных и мониторинг задач.
• Работа с CDC и репликацией данных в BigQuery, обеспечение качества данных.

Skills

GCP / Data & Analytics

BigQuery SQL

Python (Pandas)

ETL/ELT design

Data governance

Data Lakehouse

CDC Datastream

Education

Relevant degree in CS/Engineering

Tools

Cloud Dataflow

Dataplex

Pub/Sub

Cloud Composer

Datastream

Job description

Описание:

For our client in the fuel and energy sector, we are looking for a Data Engineer to join the team responsible for developing and maintaining a CDP platform based on a Data Lakehouse architecture in Google Cloud Platform.

Задачи:

• Supporting the project and maintenance team in building and developing a CDP platform using a Data Lakehouse architecture (GCP)
• Designing, implementing and optimizing ETL/ELT pipelines (batch and streaming)
• Creating and managing data transformations across the Bronze, Silver and Gold layers (medallion architecture)
• Orchestrating data flows and monitoring jobs
• Working with CDC mechanisms and replicating data from transactional sources to BigQuery
• Ensuring data quality, cataloguing metadata and complying with Data Governance principles, including GDPR
• Working closely with IT teams and technical Project Managers in an international environment

Требования:

• Experience in at least several cloud solution implementation projects in a Data Engineer role
• At least 6 months of practical experience working with GCP (Google Cloud Platform) technologies
• Very good knowledge of key GCP Data & Analytics services: BigQuery (partitioning, clustering, external tables, query optimization), Cloud Storage, Cloud Dataflow (Apache Beam) and Cloud Composer (Apache Airflow)
• Knowledge of services supporting data integration and management: Datastream (CDC/replication), Dataplex (metadata, Data Quality, Data Lineage), Pub/Sub (streaming) and Dataform (SQL transformations in BigQuery)
• Knowledge of SQL (BigQuery dialect) and Python, including Pandas, for data transformation and analysis
• Experience implementing and maintaining medallion architecture (Bronze/Silver/Gold), ETL/ELT processes and Data Governance principles, including data deduplication and versioning
• Knowledge of Git and CI/CD processes (e.g. Cloud Build), as well as Cloud Monitoring/Logging configuration
• Very good communication and organizational skills, and knowledge of English sufficient to work in an international environment

Условия:

• Work at a renowned organization (ITwiz BEST100) operating according to international quality and security standards (ISO 9001, ISO 27001)
• A non-corporate atmosphere and a flat organizational structure
• The opportunity for continuous development through: Working with a variety of technologies on projects for renowned clients
• Collaborating with an experienced team of experts who are happy to share their knowledge
• Funding for training and certifications
• Employee Referral Program
• An organizational culture based on the following values: We act with good intentions We share knowledge We take responsibility We continuously improve People are our top priority We act with an understanding of the purpose and meaning of our work We communicate honestly and professionally
• Taking part in charitable initiatives, including working with the Joanna Radziwiłł "Caring Wings" Foundation and participating in the Christmas "Gwiazdorzymy" campaign
• A diverse and inclusive work environment - representatives of four generations work together at our company
• Equal development opportunities - women hold 50% of managerial and leadership positions