Data Engineer

Taleo Consulting; Brussel Hoofdstad; Sur place; EUR 50 000 - 70 000; Plein temps

Résumé du poste

A technology consulting firm in Brussels is looking for a skilled data professional. You will be responsible for designing, developing, and maintaining scalable data pipelines using the Databricks platform. The ideal candidate has a Bachelor’s degree in a relevant field and at least 3 years of experience in SQL and data warehousing, along with expertise in Databricks and Dataiku. Strong analytical, communication, and collaboration skills are essential for this role.

Qualifications

• Minimum 3 years of experience in SQL and Data warehousing.
• Minimum 2 years of experience with Databricks.

Responsabilités

• Design, develop and maintain scalable data pipelines on the Databricks platform.
• Manage data ingestion processes from various sources into the Databricks environment.
• Implement and optimize ETL processes to ensure efficient data transformation and integration.
• Ensure data quality, integrity, and security across all data pipelines.
• Monitor and troubleshoot data pipeline performance, implementing necessary maintenance and updates.
• Document data pipeline processes and technical specifications.
• Realize knowledge transfer towards the Data engineer team.

Connaissances

Strong skills in SQL

Programming languages like Python

Datawarehouse / Lakehouse projects

Proven expertise in Databricks

Dataiku

AWS services

Analytical mindset

Strong communication skills

Collaboration skills

Formation

Bachelor’s degree in Computer Science

Engineering

Mathematics

Related field

Outils

Databricks

Dataiku

AWS

Description du poste

• Design, develop and maintain scalable data pipelines on the Databricks platform
• Manage data ingestion processes from various sources into the Databricks environment.
• Implement and optimize ETL (Extract, Transform, Load) processes to ensure efficient data transformation and integration.
• Ensure data quality, integrity, and security across all data pipelines.
• Monitor and troubleshoot data pipeline performance, implementing necessary maintenance and updates.
• Document data pipeline processes, technical specifications and best practices.
• Realize knowledge transfer towards the Data engineer team which will take over the responsibility to run the platform in the future
• You demonstrate and use your developer experience to develop and maintain complex data transformation steps using Dataiku under specific guidelines (dev vs automation node, use of Dataiku’s low code recipes, etc.)

Required profile.

Required Skills

• Strong skills in SQL (and other relevant programming languages like Python) and in Datawarehouse / Lakehouse projects
• Proven expertise in Databricks, Dataiku and with AWS services
• Analytical mindset with attention to detail
• Strong communication and collaboration skills (within a Project team and with a Data engineer team

Qualifications

• Bachelor’s degree in Computer Science, Engineering, Mathematics, or a related field.
• Minimum 3 years of experience in SQL and Data warehousing
• Minimum 2 years of experience with Databricks

We are always on the lookout for talented people to enrich our culture and growing family.