Data Engineer - Lead Level

emagine Consulting; France; À distance; EUR 60 000 - 90 000; Plein temps

Résumé du poste

A leading consulting firm in the banking industry seeks a Lead Data Engineer. In this role, you will decommission a legacy CRM system and migrate to a new platform, leveraging your strong background in SQL, NoSQL, and GCP. You will be involved in building robust data pipelines, engaging in Agile processes, and promoting development standards. Ideal candidates will have a proactive attitude and leadership experience in dynamic environments.

Qualifications

• Strong background in SQL and NoSQL databases.
• Proficiency in Python and GCP development.
• Experience in a Technical Lead position with Agile and DevOps methodologies.

Responsabilités

• Build data pipelines using BigQuery, DataProc, Python, Composer, and Spark.
• Participate in Sprint reviews and retrospectives.
• Engage in software design and automated testing.

Connaissances

SQL

NoSQL

Python

GCP

Agile

DevOps

Data Warehousing

Description du poste

Industry : Banking

The project is focused on decommissioning a legacy CRM system and successfully migrating to a new platform by the end of the year. The main function of the Lead Data Engineer is to support the development of robust data pipelines and ensure smooth data migration and integration.

Main Responsibilities

Be a self-starter and proactive, experienced Technical Lead.

Exhibit excellent communication, interpersonal, and decision-making skills.

Participate and influence in regular planning and status meetings.

Provide input to the development process through involvement in Sprint reviews and retrospectives.

Contribute to system architecture and design.

Utilize good experience with SQL and NoSQL databases.

Engage in software design, Python & GCP development , and automated testing of new and existing components in an Agile, DevOps, and dynamic environment.

Promote development standards, conduct code reviews, and facilitate knowledge sharing.

Build data pipelines using technologies such as BigQuery, DataProc, Python, Composer, and Spark.

Demonstrate knowledge and experience of the GCP ecosystem and data management frameworks.

Have CI / CD experience .

Possess knowledge of Software Development Life Cycle (SDLC) and methodologies like DevOps, Agile, Scrum, Waterfall, and Iterative process.

Show good knowledge of Data Warehouse concepts.

Implement tools and processes while handling performance, scale, availability, accuracy, and monitoring.

Key Requirements

Experience in a Technical Lead position.

Strong background in SQL and NoSQL databases.

Proficiency in Python and GCP development.

Experience with CI / CD processes.

Solid understanding of Agile and DevOps methodologies.

Good knowledge of Data Warehouse concepts.

Nice to Have

Knowledge and experience of Hadoop.

Familiarity with PySpark, Scala, Hive, and Java.