Remote Google Cloud Data Engineer - ETL/Streaming Expert

Itelence; Polska; Remote; PLN 179,000 - 220,000; Full time

Job summary

ITELENCE poszukuje specjalisty Data Engineer do długoterminowego, zdalnego projektu dla międzynarodowych klientów. Praca w modelu B2B, 100% zdalnie, Full-time.

Kluczowe zadania to projektowanie i utrzymanie pipeline'ów danych w Google Cloud Platform, implementacja ETL/ELT w Pythonie i SQL oraz pracy z BigQuery i przetwarzaniem danych w czasie rzeczywistym. Wymagane minimum 3 lata doświadczenia, znajomość GCP, Python, SQL, BigQuery, Pub/Sub, Dataflow, Airflow/Composer i dbt/Dataform.

Qualifications

• Minimum 3 years of commercial experience as Data Engineer and practical experience with Google Cloud Platform.
• Practical experience with Python and SQL.
• Experience with BigQuery and other GCP services.
• Experience with Cloud Storage, Cloud Run, Cloud Workflows and Cloud Composer / Apache Airflow.
• Knowledge of real-time processing solutions and event-driven architecture.
• Hands-on experience with Pub/Sub, Dataflow / Apache Beam, Kafka / Confluent, Eventarc.
• Experience building and maintaining scalable data pipelines for large data volumes and high event rates.
• Familiarity with data transformation and orchestration tools, especially dbt, Dataform and Apache Airflow.
• Ability to work in production data environments and address performance and reliability issues.
• Very good English language skills for international collaboration.

Responsibilities

• Design, develop and maintain scalable data pipelines in Google Cloud Platform.
• Build and develop ETL/ELT processes using Python and SQL.
• Work with BigQuery, including real-time data processing solutions.
• Design and implement streaming and event-driven data processing solutions.
• Build pipelines using Pub/Sub, Dataflow / Apache Beam, Kafka / Confluent and Eventarc.
• Design and maintain orchestration processes using Apache Airflow / Cloud Composer.
• Develop data transformation using dbt and Dataform.
• Monitor performance, quality and reliability of pipelines and identify optimization opportunities.
• Collaborate with Data Engineering, Analytics and other technical teams to design and implement data solutions.
• Ensure scalability, security and reliability of production data architectures.

Skills

Python

SQL

Google Cloud Platform

BigQuery

Pub/Sub

Dataflow

Apache Beam

Kafka

Confluent

Eventarc

Apache Airflow

Cloud Composer

dbt

Dataform

Terraform

Docker

CI/CD

Tools

Google Cloud Platform

BigQuery

Pub/Sub

Dataflow

Apache Beam

Kafka

Confluent

Eventarc

Apache Airflow

Cloud Composer

dbt

Dataform

Terraform

Docker

CI/CD

Job description

ITELENCE poszukuje specjalisty Data Engineer do długoterminowego, zdalnego projektu dla międzynarodowych klientów. Praca w modelu B2B, 100% zdalnie, Full-time.

Kluczowe zadania to projektowanie i utrzymanie pipeline'ów danych w Google Cloud Platform, implementacja ETL/ELT w Pythonie i SQL oraz pracy z BigQuery i przetwarzaniem danych w czasie rzeczywistym. Wymagane minimum 3 lata doświadczenia, znajomość GCP, Python, SQL, BigQuery, Pub/Sub, Dataflow, Airflow/Composer i dbt/Dataform.