Cloud Data Engineer: Pipelines & ETL Quality

Magellan; Lille; Hybride; EUR 50 000 - 70 000; Plein temps

Résumé du poste

Magellan is seeking a Data Engineer to industrialise data platforms (pipelines, ETL, quality, governance) on GCP or AWS cloud environments. You will coordinate with Data Scientists and Analysts to design robust, scalable data workflows that power analytics and decision making.

Apply CI/CD for data pipelines, manage SQL and NoSQL stores, ensure data quality and interoperability across sources, and continuously optimise flows for performance and reliability in a cloud setting.

Qualifications

• Design and develop robust data pipelines: collection, processing, storage.
• Optimise data flows to improve efficiency and performance.
• Manage SQL and NoSQL database infrastructures in a GCP or AWS Cloud environment following CI/CD principles.
• Integrate data by developing ETL processes and ensuring interoperability between different systems and sources.
• Guarantee data quality: implement quality and consistency checks, cleansing and validation processes.
• Liaise with the entire Data team (Scientists and Analysts) to support and address business needs.

Responsabilités

• Design and develop robust data pipelines: collection, processing, storage.
• Optimise data flows to improve efficiency and performance.
• Manage SQL and NoSQL database infrastructures in a GCP or AWS Cloud environment following CI/CD principles.
• Integrate data by developing ETL processes and ensuring interoperability between different systems and sources.
• Guarantee data quality: implement quality and consistency checks, cleansing and validation processes.
• Liaise with the entire Data team (Scientists and Analysts) to support and address business needs.

Connaissances

Data engineering

ETL

SQL

NoSQL

GCP

AWS

CI/CD

Data quality

Outils

GCP

AWS

Description du poste

Magellan is seeking a Data Engineer to industrialise data platforms (pipelines, ETL, quality, governance) on GCP or AWS cloud environments. You will coordinate with Data Scientists and Analysts to design robust, scalable data workflows that power analytics and decision making.

Apply CI/CD for data pipelines, manage SQL and NoSQL stores, ensure data quality and interoperability across sources, and continuously optimise flows for performance and reliability in a cloud setting.