Databricks Data Engineer: Build & Optimize Pipelines

Degetel; Brussel; Sur place; EUR 60 000 - 90 000; Plein temps

Résumé du poste

Degetel in Belgium is seeking a data engineer to implement, design, build and maintain scalable data pipelines on Databricks. You will manage data ingestion from various sources and optimize ETL processes.

You will ensure data quality, integrity and security across pipelines, monitor performance and document technical specs. You will transfer knowledge to the on-going run team and collaborate across project and run teams.

Qualifications

• Bachelor's degree in a related field required.
• Minimum 4 years of SQL/Python experience and 4 years with Databricks.
• 2 years in Data Warehousing.

Responsabilités

• Implement, design, build and maintain scalable data pipelines on Databricks.
• Manage data ingestion from various sources.
• Implement and optimize ETL processes.
• Ensure data quality, integrity and security across pipelines.
• Monitor and troubleshoot pipeline performance.
• Document processes, technical specs and best practices.
• Transfer knowledge to the team that will take over running the platform.

Connaissances

SQL

Python

Analytical mindset

Detail-oriented

Communication skills

Formation

Bachelor's degree in Computer Science, Engineering, Mathematics or a related field

Outils

Databricks

AWS

Description du poste

Degetel in Belgium is seeking a data engineer to implement, design, build and maintain scalable data pipelines on Databricks. You will manage data ingestion from various sources and optimize ETL processes.

You will ensure data quality, integrity and security across pipelines, monitor performance and document technical specs. You will transfer knowledge to the on-going run team and collaborate across project and run teams.