Senior Data Engineer

Ubique Systems; Brussel Hoofdstad; Sur place; EUR 70 000 - 100 000; Plein temps

Résumé du poste

Ubique Systems is seeking a Senior Data Engineer to design scalable data pipelines using Python, PySpark, Databricks, and AWS. The candidate will transform and validate data, ensure quality, and support production data platforms.

The role involves building ETL/ELT processes, developing workflows with Apache Airflow, and collaborating on data architecture and CI/CD pipelines using Azure DevOps/TFS, with Terraform for infra as code.

Qualifications

• Senior Data Engineer with hands-on experience in Python, PySpark, Databricks, and AWS.
• Develop scalable data pipelines and transform structured/unstructured data.
• Ensure data quality and support production data platforms.

Responsabilités

• Design, develop, and maintain scalable data pipelines and ETL/ELT processes.
• Ingest, integrate, map, cleanse, transform, and validate data.
• Develop data processing solutions using Python, PySpark, and Databricks.

Connaissances

Python

PySpark

Databricks

AWS

Apache Airflow

Terraform

Azure DevOps

CI/CD pipelines

Outils

Terraform

Apache Airflow

Azure DevOps

Description du poste

Job Summary:

We are seeking a Senior Data Engineer with strong hands‑on experience in Python, PySpark, Databricks, and AWS. The candidate will be responsible for developing scalable data pipelines, integrating and transforming structured/unstructured data, ensuring data quality, and supporting production data platforms.

Key Responsibilities:

• Design, develop, and maintain scalable data pipelines and ETL/ELT processes.
• Ingest, integrate, map, cleanse, transform, and validate structured and unstructured data.
• Develop data processing solutions using Python 3, PySpark, and Databricks.
• Build and manage workflows using Apache Airflow.
• Work with AWS cloud services for data engineering solutions.
• Write technical specifications, unit tests, and maintain technical documentation.
• Participate in data architecture and technical design discussions.
• Implement data engineering and data management best practices.
• Develop and maintain CI/CD pipelines using Azure DevOps/TFS/ VSTS.
• Use Terraform for infrastructure automation where required.
• Monitor production pipelines, troubleshoot issues, and provide maintenance/support.
• Ensure timely delivery and adherence to project deadlines.