Data Engineer

Xebia; Comunidad Valenciana; Presencial; EUR 50.000 - 90.000; Jornada completa

Descripción de la vacante

An innovative consulting firm is on the lookout for a Senior Data Engineer to spearhead the design and implementation of data pipelines for a significant Loyalty program expansion. This role requires a seasoned professional with over six years of experience in data engineering, particularly in data architecture and pipeline development. You will be responsible for ensuring data integrity and optimizing performance across various systems. Join a dynamic team that values collaboration and innovation, and make a substantial impact on data-driven business decisions in a rapidly evolving digital landscape.

Formación

• 6+ years of experience in data engineering with a focus on data architecture.
• Proven experience with cloud platforms (GCP) and big data technologies.

Responsabilidades

• Lead design and implementation of data pipelines for Loyalty program expansion.
• Integrate data from various sources into the data lake and warehouse.

Conocimientos

Python

Pyspark

GCP

Airflow

Databricks

SQL

Data modelling

Schema design

Data warehousing

Problem-solving

Herramientas

Airflow

Spark

DBT

BigQuery

PostgreSQL

MySQL

MongoDB

Cassandra

Descripción del empleo

For more than 20 years, our global network of passionate technologists and pioneering craftspeople has delivered cutting-edge technology and game-changing consulting to companies on the brink of AI driven digital transformation. Since 2001, we have grown into a full service digital consulting company with 5500+ professionals working on a worldwide ambition. Driven by the desire to make a difference, we keep innovating. Fuelling the growth of our company with our knowledge worker culture. When teaming up with Xebia, expect in-depth expertise based on an authentic, value-led, and high quality way of working that inspires all we do.

About the Role

We are seeking a highly skilled and experienced onsite Senior Data Engineer to lead the design and implementation of data pipelines and data products arising out of expansion of the Loyalty program. The ideal candidate will have extensive experience building scalable data solutions, strong analytical skills, and a passion for leveraging data to drive business decisions.

1. Design and Implementation :

1. Lead the design and implementation of the Loyalty program expansion.
2. Develop and maintain data pipelines to ensure seamless data flow from the Loyalty system to the data lake and data warehouse.
3. Ensure data integrity, consistency, and availability across all data systems.

2. Data Integration :

1. Integrate data from various sources, including transactional databases, third-party APIs, and external data sources, into the data lake.
2. Implement ETL processes to transform and load data into the data warehouse for analytics and reporting.
3. Work closely with cross-functional teams including Engineering, Business Analytics, Data Science and Product Management to understand data requirements and deliver solutions.
4. Collaborate with data engineers to ensure data engineering best practices are integrated into the development process.

3. Performance Optimisation :

1. Optimise data storage and retrieval to improve performance and scalability.
2. Monitor and troubleshoot data pipelines to ensure high reliability and efficiency.

4. Data Governance :

1. Implement and enforce data governance policies to ensure data security, privacy, and compliance.
2. Develop documentation and standards for data processes and procedures.

Experience :

1. 6+ years of experience in data engineering, with a focus on data architecture and pipeline development.
2. Proven experience with cloud platforms (GCP) and big data technologies (e.g., Airflow, Spark, DBT, Databricks, BigQuery, GCP Services).
3. Hands-on experience with ETL tools and processes.
4. Strong SQL skills and experience with relational databases (e.g., PostgreSQL, MySQL) and NoSQL databases (e.g., MongoDB, Cassandra).

Skills :

1. Proficiency in programming languages such as Python.
2. Pyspark, GCP, Airflow, Databricks, SQL.
3. Experience with data modelling, schema design, and data warehousing concepts.
4. Familiarity with data lake architectures and best practices.
5. Strong problem-solving skills and attention to detail.
6. Excellent communication and collaboration skills.