Data Engineer: PySpark, Databricks & AWS for Energy

Yechte; Brussel Hoofdstad; Hybride; EUR 70 000 - 100 000; Plein temps

Avantages offerts par ce poste

Flexi-hours

Hybrid work

Mobility plan

Food allowances

Healthcare support

IT trainings & certifications

Résumé du poste

yechte in Brussels, Belgium is seeking a Data Engineer with strong experience in developing and maintaining data solutions within enterprise environments. You will design and deliver data pipelines using Python, PySpark, Databricks, and AWS, focusing on data processing, ETL, data management, and production support.

Collaborating with multidisciplinary teams, you will help industrialise business processes, support costing, exposure management, and forecasting activities for gas and power, while

Qualifications

• Proven experience as a Data Engineer or in a similar data development role.
• Strong hands-on experience with Python V3.
• Experience with PySpark and Databricks.
• Experience with AWS and SQL.
• Experience with ETL development and data transformation.
• Ability to participate in architecture discussions and solution design.
• Strong analytical and problem solving skills.

Responsabilités

• Develop and maintain data solutions within the data framework.
• Design and implement scalable data processing and transformation solutions.
• Develop data pipelines using Python and PySpark.
• Build and execute ETL processes using Databricks.
• Work with Apache Spark and manage Spark clusters.
• Develop and optimise SQL queries for efficient data processing.
• Work with relational databases and data warehouses.
• Write and maintain technical specifications.
• Write and execute unit tests to ensure solution quality.
• Participate in architecture discussions and contribute to technical solution design.
• Apply and implement best practices around data management.
• Ensure data solutions comply with agreed technical and architectural standards.
• Deliver within timelines and quality standards.
• Perform production monitoring and maintenance.
• Analyse and resolve production issues affecting data solutions.
• Collaborate with multidisciplinary teams and stakeholders.
• Translate customer requirements into effective data solutions.

Connaissances

Python

PySpark

Databricks

AWS

SQL

Data pipelines

ETL development

Spark clusters

Outils

Airflow

Azure DevOps

Terraform

Postman

Description du poste

yechte in Brussels, Belgium is seeking a Data Engineer with strong experience in developing and maintaining data solutions within enterprise environments. You will design and deliver data pipelines using Python, PySpark, Databricks, and AWS, focusing on data processing, ETL, data management, and production support.

Collaborating with multidisciplinary teams, you will help industrialise business processes, support costing, exposure management, and forecasting activities for gas and power, while