Remote Data Platform Engineer - Spark, SQL & Cloud

bridge351; Lisboa; Presencial; EUR 65 000 - 90 000; Tempo integral

Vantagens oferecidas por esta oferta de emprego

Health and Life Insurance

100% Remote opportunities

Tech Visa Support

Resumo da oferta

bridge351 is seeking a Data Engineer to support and develop a data platform for a large-scale international energy provider. You will work remotely from Portugal, provide user support, troubleshoot issues, and help build reliable data pipelines and ETL/ELT processes.

You will collaborate with platform architects to ensure the data platform is efficient, secure, and highly available. You will improve data processes, monitor performance, and contribute to data lakes, warehouses, and streaming

Qualificações

• Experience with data pipelines in a production environment.
• Strong hands-on knowledge of SQL and distributed data processing (e.g., Apache Spark).
• Proficiency and knowledge of troubleshooting dbt, Apache Kafka, Apache Ignite, Apache NiFi and Apache AirFlow.
• Proficiency in a programming language such as Python or Scala for data engineering tasks.
• Experience with cloud data services (AWS).
• Familiarity with Grafana tools.
• Knowledge of data governance, security, and compliance in a cloud environment.
• Experience implementing CI/CD and testing frameworks for data workflows.
• Strong problem-solving skills, communication skills, and collaboration.

Responsabilidades

• Support and develop a data platform for a large-scale international energy provider.
• Provide user support for the data platform, troubleshooting and fixing issues.
• Support data pipelines and ETL/ELT processes to enable reliable and scalable data flows.
• Collaborate with platform architects to ensure the data platform is efficient, secure, and highly available.
• Improve data processes, monitoring, performance and platform efficiency.
• Support data quality frameworks and monitoring to ensure accuracy, consistency, and completeness of data pipelines.
• Support for data lakes, data warehouses, and data streaming solutions.
• Work with stakeholders to solve issues in the data platform.
• Automate repeatable tasks and improve data ingestion, transformation, and delivery processes.
• Troubleshoot and optimize data workflows and SQL queries for performance and cost efficiency.
• Document data platform changes, processes, and best practices.
• Review, evaluate and approve data product releases to production.

Conhecimentos

SQL

Apache Spark

dbt

Apache Kafka

Apache Ignite

Apache NiFi

Apache AirFlow

Python

Scala

AWS

Grafana

Data governance

CI/CD

Testing frameworks

Communication skills

Ferramentas

dbt

Apache Kafka

Apache NiFi

Apache AirFlow

Descrição da oferta de emprego

bridge351 is seeking a Data Engineer to support and develop a data platform for a large-scale international energy provider. You will work remotely from Portugal, provide user support, troubleshoot issues, and help build reliable data pipelines and ETL/ELT processes.

You will collaborate with platform architects to ensure the data platform is efficient, secure, and highly available. You will improve data processes, monitor performance, and contribute to data lakes, warehouses, and streaming