Databricks Data Engineer: ETL Pipelines in AWS (Belgium)

emagine; Brussel Hoofdstad; Sur place; EUR 55 000 - 85 000; Plein temps

Résumé du poste

emagine is seeking a Data Engineer to design, develop, and maintain scalable data pipelines on the Databricks platform in a Belgium-based environment. The role focuses on data ingestion, ETL optimization, and ensuring data quality and security within an AWS context.

The ideal candidate will have strong SQL and Python skills, Databricks expertise, and experience with Kimball modeling. Collaboration with the project team and knowledge transfer are essential parts of the role.

Qualifications

• Strong skills in SQL and Python.
• Proven expertise in Databricks and AWS services.
• K.modal expertise in Databricks to support BI activities (Kimball modeling).
• Analytical mindset with attention to detail.
• Strong communication and collaboration skills.

Responsabilités

• Design, develop, and maintain scalable data pipelines on the Databricks platform.
• Manage data ingestion processes from various sources into the Databricks environment.
• Implement and optimize ETL (Extract, Transform, Load) processes to ensure efficient data transformation and integration.
• Ensure data quality, integrity, and security across all data pipelines.
• Monitor and troubleshoot data pipeline performance, implementing necessary maintenance and updates.
• Document data pipeline processes, technical specifications, and best practices.
• Conduct knowledge transfer to the Data Engineer team responsible for managing the platform in the future.

Connaissances

SQL

Python

Databricks

AWS

Formation

Bachelor's degree in Computer Science/Engineering/Mathematics

Description du poste

emagine is seeking a Data Engineer to design, develop, and maintain scalable data pipelines on the Databricks platform in a Belgium-based environment. The role focuses on data ingestion, ETL optimization, and ensuring data quality and security within an AWS context.

The ideal candidate will have strong SQL and Python skills, Databricks expertise, and experience with Kimball modeling. Collaboration with the project team and knowledge transfer are essential parts of the role.