Senior Data Engineer

Nimber; Lisboa; Híbrido; EUR 70 000 - 100 000; Tempo integral

Resumo da oferta

Nimber is seeking a Senior Data Engineer to join our IT team in a hybrid model across Lisbon or Porto. You will lead data migrations from SQL Server to GCP, design scalable pipelines on BigQuery, and automate tasks with Python while collaborating with analysts and end-users.

The role requires deep GCP expertise, strong SQL and Python skills, and excellent English communication. Join us to shape a modern data platform and drive impactful decisions.

Qualificações

• Mastery of Google Cloud Platform (GCP).
• BigQuery: schema design, complex queries, cost management.
• Cloud Composer: DAGs creation, scheduling, debugging.
• SQL: strong skills, SQL Server familiarity a plus.
• Python: scripting for migration and automation.
• Data migration from on prem to cloud (SQL Server to GCP).
• English proficiency at B2 level or higher.
• Autonomy, rigor and teamwork in a new environment.

Responsabilidades

• Participate in data and ETL migration from SQL Server to GCP.
• Design and maintain ETL/ELT pipelines on GCP with Cloud Composer.
• Develop complex BigQuery queries and optimized data schemas.
• Create Python scripts for migration, automation and integration.
• Ensure data quality and monitor pipelines during migration.
• Collaborate with SQL Server admins, analysts and end-users.
• Review code, testing strategies and keep pipelines reliable.

Conhecimentos

GCP

BigQuery

Cloud Composer

SQL

Python

Data Migration

Analytical skills

English (B2)

Ferramentas

Power BI

Descrição da oferta de emprego

Hey there, think you stumbled upon this job posting by chance? We dont believe in chance at Nimber.

Caught your eye? Great. Keep going...

We are Nimber, and we are not just filling positions, we are building a team that is ready to shake things up. If you are ready to rewrite the rules and make a real impact, this is your moment. Join us and lets put the future where it belongs. In the hands of passionate people.

But what does Nimber mean?In mathematics and game theory, they are numbers with special properties of addition and multiplication. And what we really want is to add value to customers' projects.

Position: SeniorData Engineer

Location: Lisbon or Oporto - Hybrid

Job Summary: Position for Google Cloud Platform (GCP) Senior Data Engineer to join the IT (Information Technologies) department, to work in a hybrid model in the Porto or Lisbon office.

Key Responsibilities:

Data and Database Migration:

• Actively participate in the design and execution of the data and ETL process migration strategy from SQL Server to GCP.
• Develop and optimize scripts and pipelines for efficiently transferring and transforming data into BigQuery.

Design and Development of Data Pipelines on GCP:

• Design, develop, and maintain robust and high-performing ETL/ELT pipelines on GCP, replacing or rebuilding existing flows.
• Orchestrate and schedule these pipelines using Google Cloud Composer to ensure reliable migration and operationalization.

Database Management and Optimization on GCP:

• Develop and optimize complex SQL queries for BigQuery, taking into account the specificities of migrating from SQL Server to BigQuery (e.g., optimizing partitioning, clustering).
• Contribute to the design and implementation of optimized data schemas for BigQuery, adapted to new cloud architectures.

Development and Automation:

• Use Python to develop scripts for automating migration tasks, data transformation, and integration with other GCP services.
• Implement processes to ensure data quality and integrity during and after migration.

Collaboration and Support:

• Collaborate closely with current teams (SQL Server administrators, analysts) to understand the existing environment and facilitate the transition.
• Exchange with end-users (business lines, data scientists) to understand their needs and ensure the new platform meets their expectations.
• Participate in code reviews, development best practices, and testing strategies for the new platform.
• Ensure monitoring, debugging, and incident resolution related to data pipelines during the transition phase and upon production deployment.

Technological Watch and Optimization:

• Stay informed about best practices for migration and the latest developments on Google Cloud Platform.
• Propose improvements and optimizations for the new Data platform on GCP.
  

Requirements:

• Mastery of Google Cloud Platform (GCP).
• BigQuery: Significant experience in schema design, writing complex queries (SQL), performance optimization, and cost management in a cloud context.
• Cloud Composer: Solid experience in creating, scheduling, monitoring, and debugging DAGs to orchestrate data flows, particularly for migration and integration processes.
• SQL: Excellent mastery of the SQL language. Experience with SQL Server is a plus, but more importantly, the ability to adapt and optimize SQL for BigQuery.
• Python: In-depth mastery of the Python language for developing scripts (migration, automation, data transformation) and interacting with GCP APIs.
• Data Migration: Experience or strong understanding of the challenges and methodologies for migrating databases from an on-premise environment (SQL Server) to a cloud environment (GCP).
• Analytical and Problem-Solving Skills: Ability to analyze existing systems (SQL Server), understand business needs, design solutions on GCP, and resolve complex problems related to migration and pipeline operation.
• Good communication skills in English (B2 level required).
• Autonomy, rigor, organizational skills, teamwork ability, technical curiosity, and the capacity to adapt to a new environment.

Additional Skills (Good to Have):

• Direct knowledge of SQL Server (architecture, T-SQL, basic administration).
• Experience with other GCP services relevant to data migration and integration (Storage, Dataflow, etc.).
• Knowledge of GCP-specific migration tools and strategies (e.g., Database Migration Service).
  Experience with data visualization tools (Power BI).
• Understanding of Data Warehousing concepts and data modeling (Data Vault).
• Experience with code versioning.

This is our winning strategy, built on the core principles of impartiality, independence, and fairness at Nimber. The only thing missing is you!