Data Engineer: Big Data Pipelines (Spark/Scala) Hybrid

BNP Paribas Personal Finance; Madrid; Híbrido; EUR 45.000 - 70.000; Jornada completa

Ventajas ofrecidas por este puesto de trabajo

Training programs, career plans

Internal mobility opportunities

Hybrid telecommuting model (50%)

31 vacation days

Descripción de la vacante

BNP Paribas Personal Finance is seeking a Data Engineer - Developer in SET Iberia to design and maintain data pipelines and platforms across multiple sources. You will work on Spark-based processing, Hadoop-to-cloud migrations, and data quality at scale, enabling robust data products for global BNPP PF entities.

You will join a diverse IT and Data team, collaborating to deliver end-to-end data solutions with a hybrid work model and strong emphasis on security and performance.

Formación

• Data Modeling and Pipelines development with Spark and Scala to ingest and transform data.
• Manage big data storage solutions (HDFS, IBM Cloud Storage, databases).
• Data Transformation and Quality: implement data transformation and quality controls.
• CI/CD Pipeline Implementation for deployment, tests and development management.
• Infrastructure Migration to cloud using Kubernetes Engine and Airflow.

Responsabilidades

• Implement schemas, queries, and views in SQL/NoSQL databases (MongoDB, Oracle, Postgres).
• Develop scheduled data pipelines with shell scripts and Airflow.
• Validate testing to ensure data accuracy and integrity.
• Document specifications and operational runbooks for knowledge sharing.
• Improve code for performance and maintainability; ensure security practices.

Conocimientos

Spark & Scala

Kafka

HDFS

SQL/NoSQL

Airflow

Dremio

IBM Cloud Storage

Parquet format

MongoDB

Oracle

Herramientas

Spark on Scala

Spark Streaming

Airflow as a service

Kubernetes Engine

IBM Cloud Object Storage

Parquet

Descripción del empleo

BNP Paribas Personal Finance is seeking a Data Engineer - Developer in SET Iberia to design and maintain data pipelines and platforms across multiple sources. You will work on Spark-based processing, Hadoop-to-cloud migrations, and data quality at scale, enabling robust data products for global BNPP PF entities.

You will join a diverse IT and Data team, collaborating to deliver end-to-end data solutions with a hybrid work model and strong emphasis on security and performance.