Data Engineer

HN Services España; Madrid; Hybrid; EUR 60,000 - 90,000; Full time

Benefits offered by this job

Career plan tailored to employee

Progression within the company

Flexible working hours

Job summary

HN Services España in Madrid is seeking a Data Engineer to join a stable, international project. You will contribute to production support and maintenance of the DataHub, resolve data incidents, and develop scalable data pipelines using Scala, Spark, and cloud storage.

You will work on data modeling, ETL processes, and ensure data quality and security while implementing CI/CD pipelines and testing. The role offers flexible hours and a hybrid work model in Madrid.

Qualifications

• Experience with Spark on Scala and large-scale data pipelines.
• Solid knowledge of Hadoop and HDFS with SQL databases.
• Experience with cloud object storage and data virtualization (Dremio).
• Familiarity with SDLC and Agile ceremonies.

Responsibilities

• Contribute to production support and maintenance of the DataHub.
• Develop scalable data pipelines and perform data transformations.
• Ensure data quality, security, and performance of the data infrastructure.
• Configure CI/CD pipelines and write tests.
• Migrate Hadoop to cloud-based, containerized environments.

Skills

Spark Scala

HDFS

SQL

Hadoop

S3 COS

Shell scripting

SDLC awareness

Agile

Dremio

English B2

Tools

Airflow

Kubernetes Engine

COS

Job description

We are an international technology services company founded in 1983 and currently have over 2,000 employees in 5 countries: France, Spain, Romania, Portugal, and Luxembourg!

What are we looking for?

A Data Engineer to join a stable international project, based in Madrid.

Responsibilities:

• Contribute to the production support and maintenance of the DataHub, correct incidents and anomalies, resolve data-related issues, and implement functional and technical developments to ensure the stability of production processes and ensure the DataHub is available to users with minimal disruption during agreed uptime requirements.
• Modify existing code according to business requirements and continuously improve it for improved performance and maintainability.
• Ensure the performance and security of the data infrastructure and follow data engineering best practices.
• Data modeling and development of efficient data pipelines to enrich and transform large volumes of data with complex business rules, automate data pipelines, and optimize data ingestion. Design and implement scalable and secure data processing pipelines using Scala, Spark, cloud object storage, Hadoop, and COS.
• Integrate data from multiple sources and formats into the raw layer of the DataHub.
• Implement data transformation and quality to ensure data consistency and accuracy. Use programming languages such as Scala and SQL and tools such as Spark for data transformation and enrichment operations.
• Configure CI/CD pipelines to automate deployments, unit testing, and development management.
• Write and perform unit and validation tests to ensure the accuracy and integrity of the developed code.
• Implement various orchestrators and scheduling processes to automate data pipeline execution (Airflow as a Service).
• Migrate existing Hadoop infrastructure to cloud infrastructure on Kubernetes Engine, COS, Spark as a Service, and Airflow as a Service.
• Write technical documentation (specifications, operational documents) to ensure knowledge capitalization.

Requirements:

• Spark on Scala
• HDFS and structured databases (SQL)
• Hadoop
• S3 storage/COS
• Shell script
• Software Development Lifecycle (SDLC) awareness
• Agile principles and ceremonies
• Dremio as tool to virtualize data

Optionally/ as a plus

• Elasticsearch and Kibana
• HVault
• English B2 or higher.

Work Model:

• Hibrid.
• Flexible hours, Monday to Friday.

We offer:

• Career plan tailored to employee preferences
• Progression within the company
• Flexible working hours

Would you like to join our team?

If you have experience in data and are looking to grow technically and professionally, don't hesitate to apply for this position. Contact us!