Mid Data Platform Engineer

bridge351; Lisboa; Presencial; EUR 65 000 - 90 000; Tempo integral

Vantagens oferecidas por esta oferta de emprego

Health and Life Insurance

100% Remote opportunities

Tech Visa Support

Resumo da oferta

bridge351 is seeking a Data Engineer to support and develop a data platform for a large-scale international energy provider. You will work remotely from Portugal, provide user support, troubleshoot issues, and help build reliable data pipelines and ETL/ELT processes.

You will collaborate with platform architects to ensure the data platform is efficient, secure, and highly available. You will improve data processes, monitor performance, and contribute to data lakes, warehouses, and streaming

Qualificações

• Experience with data pipelines in a production environment.
• Strong hands-on knowledge of SQL and distributed data processing (e.g., Apache Spark).
• Proficiency and knowledge of troubleshooting dbt, Apache Kafka, Apache Ignite, Apache NiFi and Apache AirFlow.
• Proficiency in a programming language such as Python or Scala for data engineering tasks.
• Experience with cloud data services (AWS).
• Familiarity with Grafana tools.
• Knowledge of data governance, security, and compliance in a cloud environment.
• Experience implementing CI/CD and testing frameworks for data workflows.
• Strong problem-solving skills, communication skills, and collaboration.

Responsabilidades

• Support and develop a data platform for a large-scale international energy provider.
• Provide user support for the data platform, troubleshooting and fixing issues.
• Support data pipelines and ETL/ELT processes to enable reliable and scalable data flows.
• Collaborate with platform architects to ensure the data platform is efficient, secure, and highly available.
• Improve data processes, monitoring, performance and platform efficiency.
• Support data quality frameworks and monitoring to ensure accuracy, consistency, and completeness of data pipelines.
• Support for data lakes, data warehouses, and data streaming solutions.
• Work with stakeholders to solve issues in the data platform.
• Automate repeatable tasks and improve data ingestion, transformation, and delivery processes.
• Troubleshoot and optimize data workflows and SQL queries for performance and cost efficiency.
• Document data platform changes, processes, and best practices.
• Review, evaluate and approve data product releases to production.

Conhecimentos

SQL

Apache Spark

dbt

Apache Kafka

Apache Ignite

Apache NiFi

Apache AirFlow

Python

Scala

AWS

Grafana

Data governance

CI/CD

Testing frameworks

Communication skills

Ferramentas

dbt

Apache Kafka

Apache NiFi

Apache AirFlow

Descrição da oferta de emprego

• Fluent in written and spoken Portuguese and English.
• 30 days’ notice period for termination
• The candidate can work remotely from any Portuguese region
  

• Fluent in written and spoken Portuguese and English.
• 30 days’ notice period for termination
• The candidate can work remotely from any Portuguese region
  

What you will be doing:

• We're looking for a Data Engineer to support and develop a data platform within a large-scale international energy provider.
• Provide user support for the data platform, troubleshooting and fixing issues
• Support data pipelines and ETL/ELT processes to enable reliable and scalable data flows
• Collaborate with platform architects to ensure the data platform is efficient, secure, and highly available
• Improve data processes, monitoring, performance and platform efficiency
• Support data quality frameworks and monitoring to ensure accuracy, consistency, and completeness of data pipelines
• Support for data lakes, data warehouses, and data streaming solutions
• Work with stakeholders (data scientists, analysts, business users) to solve issues in the data platform
• Automate repeatable tasks and improve the data ingestion, transformation, and delivery processes
• Troubleshoot and optimize data workflows and SQL queries for performance and cost efficiency
• Document data platform changes, processes, and best practices
• Review, evaluate and approve data product releases to production.

What we are looking for:

• Experience with data pipelines in a production environment
• Strong hands-on knowledge of SQL and experience with distributed data processing (e.g., Apache Spark)
• Proficiency and knowledge of troubleshooting dbt, Apache Kafka, Apache Ignite, Apache NiFi and Apache AirFlow
• Proficiency in a programming language such as Python or Scala for data engineering tasks
• Experience with cloud data services (AWS)
• Familiarity with Grafana tools
• Knowledge of data governance, security, and compliance in a cloud environment
• Experience implementing CI/CD and testing frameworks for data workflows
• Strong problem-solving skills, communication skills, and passion for collaboration.

What can you expect from us?

Mind-blowing workplace culture. You will be integrated in a professional, dynamic and collaborative team.

100% Remote opportunities

We want you to have the flexibility to work where you feel most comfortable and productive.

International Career

• You can expect professional growth and to be connect with the world.
• We are represented in Portugal, Belgium, Luxembourg, and Denmark.
• And with projects in many other countries: Netherlands, Luxembourg, Singapore and in the United States of America (and a lot more is coming…)

Extra Benefits & Perks

If you wish to work with us and you are outside European Union (good news…) we are a Tech Visa Company, We will help!

As a plus, we provide Health and Life Insurance.