GCP Data Engineer: BigQuery Pipelines & Migration

Decskill; Porto; Presencial; EUR 60 000 - 90 000; Tempo integral

Resumo da oferta

Decskill is seeking a Data Engineer specializing in Google Cloud Platform to lead the migration of data and ETL processes from SQL Server to GCP. You will design, build, and optimize pipelines on BigQuery, orchestrated via Cloud Composer, ensuring data quality and scalable architectures.

You will work closely with SQL Server admins, analysts, and data scientists to deliver robust data platforms, implement automation with Python, and model data for cloud-native analytics.

Qualificações

• Significant experience with Google Cloud Platform (GCP) including BigQuery design and performance optimization.
• Solid experience in creating, scheduling, monitoring, and debugging DAGs to orchestrate data flows.
• Strong SQL skills; ability to optimize queries for BigQuery and migrate from SQL Server.
• Proficient in Python for migration automation and data transformation tasks.

Responsabilidades

• Participate in design and execution of data and ETL migration from SQL Server to GCP.
• Develop and optimize pipelines for transferring and transforming data into BigQuery.
• Design and maintain robust ETL/ELT pipelines on GCP and orchestrate with Cloud Composer.
• Develop and optimize SQL queries and data schemas for BigQuery during migration.
• Use Python to automate migration tasks and ensure data quality during migration.
• Collaborate with SQL Server admins, analysts, and data scientists to meet user needs.
• Participate in code reviews, best practices, and testing strategies for the platform.
• Ensure monitoring, debugging, and incident resolution for data pipelines during transition and production.

Conhecimentos

BigQuery

Cloud Composer

SQL

Python

Data Migration

Analytical Skills

English (B2)

GCP

Spark

Java

Scala

Ferramentas

Power BI

SQL Server

Git

Dataflow

Database Migration Service

Descrição da oferta de emprego

Decskill is seeking a Data Engineer specializing in Google Cloud Platform to lead the migration of data and ETL processes from SQL Server to GCP. You will design, build, and optimize pipelines on BigQuery, orchestrated via Cloud Composer, ensuring data quality and scalable architectures.

You will work closely with SQL Server admins, analysts, and data scientists to deliver robust data platforms, implement automation with Python, and model data for cloud-native analytics.