Data Lake Developer — Spark, Scala & SQL (Remote)

Talan; Salamanca; Remote; EUR 45,000 - 65,000; Full time

Benefits offered by this job

Private medical insurance

Life insurance

Lunch and transport cards

Online language classes

Smart Office Pack

Job summary

Talan is seeking a Big Data Developer to join an Agile teamWorking on data lake solutions for Market Risk, you will design, develop and evolve large-scale data pipelines using Spark, Scala and SQL. You will ensure ingestion, transformation, validation and publication of critical data while collaborating with cross-functional teams.

You will handle batch workflows with Control-M and Airflow, optimize Spark workloads and participate in production support, with a focus on data quality, reliability

Qualifications

• : 2+ years of professional experience in Big Data, Data Lakes or cloud-based data platforms.
• : Strong hands-on experience with Apache Spark, Scala and SQL for large-scale data processing.
• : Good understanding of Data Lake architecture, data quality controls, data lineage and production-grade data pipelines.
• : Strong analytical and problem-solving skills.
• : Excellent communication and teamwork skills, with the ability to collaborate with technical and functional stakeholders.
• : Adaptability and a proactive approach to solving technical challenges.
• : English: C1 level or equivalent professional fluency.
• : Spanish is considered a plus.

Responsibilities

• : Design, develop and maintain data lake pipelines using Apache Spark, Scala and SQL.
• : Build and maintain scalable data processing solutions for large volumes of data.
• : Orchestrate batch workflows and dependencies using tools such as Control-M and Airflow.
• : Implement data ingestion, transformation, validation and publication processes for Market Risk use cases.
• : Analyse and optimise the performance, reliability and scalability of Spark-based workloads.
• : Analyse requirements, identify gaps and propose solutions.
• : Contribute to delivery planning and collaborate with teams to ensure successful implementation.
• : Support production incidents, perform root-cause analysis and drive improvement.
• : Work with Agile squads and production support teams to deliver value.

Skills

Spark

Scala

SQL

Data Lake architecture

Data pipelines

Control-M

Airflow

Cloud platforms

English (C1)

Spanish a plus

Tools

Databricks

Azure

AWS

YARN

GitHub

Jenkins

SonarQube

ServiceNow

Job description

Talan is seeking a Big Data Developer to join an Agile teamWorking on data lake solutions for Market Risk, you will design, develop and evolve large-scale data pipelines using Spark, Scala and SQL. You will ensure ingestion, transformation, validation and publication of critical data while collaborating with cross-functional teams.

You will handle batch workflows with Control-M and Airflow, optimize Spark workloads and participate in production support, with a focus on data quality, reliability