Data Engineer - Developer

Tamarind Intelligence; Madrid; Híbrido; EUR 45.000 - 65.000; Jornada completa

Ventajas ofrecidas por este puesto de trabajo

Training programs, career plans

Hybrid telecommuting model (50%)

Diversity and Inclusion Committee

Corporate volunteering (1 MillionHours

Flexible compensation plan

31 vacation days

Descripción de la vacante

BNP Paribas Group in Spain seeks a Data Engineer - Developer to join our SET Iberia IT/Data/Operations team. You will design and maintain data pipelines, ingest data from Kafka, APIs, HDFS and databases into data hubs and products.

You will work with Spark/Scala, Scala-based pipelines, and collaborate across Spain and Portugal in a hybrid work model, with opportunities to grow in a multinational environment and drive data quality and performance.

Formación

• Experience in IT international companies.
• Minimum 2 years of proven success in related positions.
• Languages: English B2/C1, French a bonus.

Responsabilidades

• Data Modeling and Pipelines development using Spark and Scala to ingest and transform data.
• Manage big data storage solutions (HDFS, IBM Cloud Storage, databases).
• Data Transformation and Quality: implement data transformation and quality controls using Spark/SQL.
• CI/CD: set up pipelines for deployment and tests.
• Infrastructure migration to cloud/Kubernetes and Airflow orchestration.
• Implement SQL/NoSQL schemas and views (MongoDB, Oracle, Postgres).
• Schedule data pipelines with shell scripting and Airflow as a service.
• Documentation and knowledge sharing.

Conocimientos

English (B2/C1)

French (bonus)

Herramientas

Spark on Scala

Kafka

Spark Streaming

Airflow

Dremio

HDFS

IBM Cloud Storage

MongoDB

PostgreSQL

Oracle

Descripción del empleo

GROUP BNP PARIBAS

BNP Paribas Group is the top bank in the European Union and a major international banking establishment. It has close to 185,000 employees in 65 countries. In Spain we are more than 5,100 employees within 13 business lines.

IDRAS

Who are we?

We are South Europe Technologies (SET Iberia); the IT, Data and Operations Shared Service Center of BNP Paribas Personal Finance, with delivery centers in Spain and Portugal, providing the best solutions to BNPP PF entities around the world such as Cetelem (specialized, between others, in financial partnership of major retailers, consumer goods companies and car dealerships).

Among Other Services, Our Portfolio Is Composed Of

• Applications Management (Architecture, Project Management, Development, and Quality Assurance)
• IT Risks & Cybersecurity Services
• Platforms Management
• Data Analytics and AI
• Operations

Our offices are in Spain (Madrid) and Portugal (Lisbon, Porto). The company brings together over 200+ employees, with expertise in various technologies (Java, .Net, Python, Tibco, APIGee) and other operational roles (Functional Analyst, Project Manager, Business Analyst, Auto Stock Financing operators). We keep growing!

Our consistent track record of services delivery means comfort for our customers and opportunities for our employees.

You will find SE.T to be full of energy and an Inclusive Workplace in which you truly can make a difference.

Would you like to join our international team that delivers end-to-end solutions (applications and operations activities) to businesses of BNP Paribas Personal Finance Group entities around the world?

In a context of maintaining the high level of existing activities while growing the number of international customers, we are looking for our Data Engineer - Developer.

Mission

ABOUT THE JOB

The data engineer is a key role in any of the different data squads in charge of development and maintenance of new big data data platforms and data products. His/her main mission is to develop the different data pipelines which ingest, transform and prepare data from different sources into the different layers of our datahub platform and/or data products.

Responsibilities

• Data Modeling and Pipelines development with Spark and Scala to ingest and transform data from different sources (Kafka topics, APIs, HDFS, structured databases, files...) into HDFS, IBM Cloud Storage (generally in parquet format) or SQL/NOSQL databases
• followign complex business rules.
• Manage big data storage solutions in the platform (HDFS, IBM Cloud Storage, structured and non-structured databases).
• Data Transformation and Quality: implement data transformation and quality control processes to ensure data consistency and accuracy. Utilize programming languages such as Scala and SQL. And libraries like Spark for data transformation and enrichment
• operations.
• CI/CD Pipeline Implementation: set up CI/CD pipelines to automate deployment, unit testing, and development management.
• Infrastructure Migration: migrate the existing Hadoop infrastructure to cloud infrastructure on Kubernetes Engine, Object Storage (IBM Cloud storage), Spark as a service on Scala (to build the data pipelines), and Airflow as a service (to orchestrate and
• schedule the data pipelines).
• Implementation of schemas,queries, and views in SQL/NOSQL databases like Oracle, Postgres or MongoDB.
• Develop and configure scheduling of data pipelines with a combination of shell scripting and AirFlow as a service.
• Validation Testing: conduct unit and validation tests to ensure accuracy and integrity.
• Documentation: write technical documentation (specifications, operational documents) to ensure knowledge capitalization.
• Code Improvement: modify the existing code as per business requirements and continuously improve for better performance and maintainability.
• Configure Dremio Data Virtualization to interface with Parquet or as a way to expose the data in the different data products.
• Performance Optimization and Security: ensure the performance and security of the data infrastructure and follow the best practices of Data engineering.

IT Tools

• Spark on Scala as legacy data pipeline development language.
• Spark as a service on Scala as data pipeline development platform.
• Experience in the design and development of streaming procesess using Spark Streaming, Spark Structure Streaming and Apache Kafka.
• Management of legacy big data storage solutions (HDFS).
• Management of big data storage solutions (IBM Cloud Object Storage and parquet format).
• Implementation of SQL/NO SQL database schemas ,queries and views (MongoDB, Oracle, Postgres).
• Shell scripting and Airflow as data pipelie scheduling solution,
• Dremio as data virtualization tool.

Requirements

• You have experience in IT international companies.
• You have minimum 2 years of proven success in related positions.
• Languages: English B2/C1, French as a bonus.

Skills

• Business Skills:
• Knowledge of banking industry and processes.
• Business/IT Relationship.
• Behavioural Skills:
• Ability to understand, explain and support change.
• Ability to Deliver / Results Driven.
• Ability to collaborate / Teamwork.
• Ability to manage a project as a bonus.
• Transversal Skills:
• Agile.

Benefits

• Training programs, career plans and internal mobility opportunities, national and international thanks to our presence in different countries.
• Diversity and Inclusion Committee that ensures an inclusive work environment. In recent years, several employee communities have been created to organize diversity and inclusion awareness actions (PRIDE, We Generations and MixCity).
• Corporate volunteering program (1 Million Hours 2 Help) in which employees can dedicate time out of their working hours to volunteer activities.
• Flexible compensation plan
• Hybrid telecommuting model (50%)
• 31 vacation days.

Diversity and inclusion commitment

BNP Paribas Group in Spain is an equal opportunity employer and proud to provide equal employment opportunity to all job seekers. We are actively committed to ensuring that no individual is discriminated against on the grounds of age, disability, gender reassignment, marriage or civil partnership status, pregnancy and maternity/paternity, race, religion or belief, sex or sexual orientation. Equity and diversity are at the core of our recruitment policy because we believe that they foster creativity and efficiency, which in turn increase performance and productivity. We strive to reflect the society we live in, while keeping with the image of our clients.

About Our Culture

We are proud to create, maintain, and develop strategic business applications for BNP Paribas Personal Finance entities worldwide, maintaining high service levels and delivering added value to our customers. In our multicultural environment, we encourage all team members to develop their talents and skills, providing a wide variety of career opportunities and internal mobility programs within local SET teams or across other entities in the Group, both in Spain and globally.

We value the experience of our employees and work to maintain a balanced environment, offering flexibility regarding work schedules and respect for personal time. Our hybrid working model is a testament to our belief that social connection enhances our daily activities.

Diversity and inclusion are among our core values, and SET is an equal opportunity employer. We are committed to ensuring employment opportunities regardless of ethnicity, beliefs, religion, nationality, age, sex, sexual orientation, marital status, or political opinions.