Data Engineer

Connect Consulting; Brussel Hoofdstad; Sur place; EUR 40 000 - 55 000; Plein temps

Résumé du poste

A leading tech firm in Belgium seeks a Junior Databricks Data Engineer to join their team managing various Data platforms. The role involves developing and maintaining Delta Live Tables pipelines, managing data access, and implementing monitoring for data pipelines. Candidates should have a Bachelor's degree in Computer Science or related fields, with proficiency in Python and SQL. This position offers an opportunity to work in a supportive team environment.

Qualifications

• Proficiency in Python and SQL required.
• Experience with Delta Lake and Delta Live Tables is necessary.
• Good organizational skills needed.

Responsabilités

• Develop and maintain Delta Live Tables (DLT) pipelines for data ingestion.
• Establish and manage table-level access controls using Unity Catalog.
• Implement monitoring and alerting for data pipelines.

Connaissances

Proficiency in Python

Proficiency in SQL

Hands-on experience with Delta Lake

Hands-on experience with Delta Live Tables

Good organizational skills

Formation

Bachelor’s degree in Computer Science

Bachelor’s degree in Engineering

Bachelor’s degree in Mathematics

Description du poste

ENGIE One BtoC Belgium set-up a Databricks instance on top of AWS Cloud platform to start migrating projects and use cases implemented in various technologies (SAS, AWS Sagemaker, BW/HANA, etc.).

We are looking for a Junior Databricks Data Engineer to join our Squad in charge of managing our different Data platforms. You will work under the supervision of the Squad Product Owner with a specific support and onboarding.

Responsibilities

• Data ingestion and Pipeline development
  • Develop and maintain Delta Live Tables (DLT) pipelines for ingesting data from various source systems
  • Ensure data quality, lineage, and monitoring are embedded in ingestion workflows (when needed)
• Access & Security Management
  • Establish and manage table-level access controls using Unity Catalog or other Databricks-native solutions.
  • Collaborate with Infrastructure team and other teams to ensure compliance with internal and external standards.
• Monitoring & Optimization
  • Implement monitoring and alerting for data pipelines
  • Be cautious with performance and costs associated with, following best practices defined
• Support the team for different types of activities, related to Infrastructure management or requests from the different teams using the Databricks platform.

Required Skills

• Proficiency in Python and SQL
• Hands-on experience with Delta Lake and Delta Live Tables.
• Good organizational skills

Qualifications

Bachelor’s degree in Computer Science, Engineering, Mathematics, or a related field.

Certifications in Databricks (Data engineer)