Remote Big Data Developer - Spark, Scala & SQL

SmartRecruiters, Inc.; Salamanca; Remote; EUR 42,000 - 64,000; Full time

Benefits offered by this job

Remote work

Career growth

Multicultural environment

Private medical insurance

Life insurance

Lunch and transport cards

Online language classes

Smart Office Pack

Job summary

Talan Spain is seeking a Big Data Developer to design, develop and evolve data lake solutions for Market Risk use cases in an Agile environment. You will work with Spark, Scala and SQL to build large-scale data pipelines and ensure ingestion, transformation and publication of critical data.

You will collaborate with Agile squads, production support and functional teams to improve performance, reliability and scalability, while supporting incidents and continuous improvement in data operations.

Qualifications

• 2+ years of professional experience in Big Data, Data Lakes or cloud-based data platforms.
• Strong hands-on experience with Apache Spark, Scala and SQL for large-scale data processing.
• Good understanding of Data Lake architecture, data quality controls, data lineage and production-grade data pipelines.
• Strong analytical and problem-solving skills.
• Excellent communication and teamwork skills, with the ability to collaborate effectively with technical and functional stakeholders.
• English: C1 level or equivalent fluency.
• Spanish is considered a plus.

Responsibilities

• Design, develop and maintain data lake pipelines using Apache Spark, Scala and SQL.
• Build and maintain scalable data processing solutions for large volumes of data.
• Orchestrate batch workflows using Control-M and Airflow.
• Implement ingestion, transformation, validation and publication processes for Market Risk use cases.
• Analyse and optimise performance, reliability and scalability of Spark workloads.
• Analyse requirements, identify gaps and propose solutions.
• Contribute to delivery planning and collaborate with teams for successful implementation.
• Support production incidents and perform root-cause analysis.
• Work closely with Agile squads and production teams to deliver value.
• Follow best practices to ensure robust, production-ready solutions.

Skills

Apache Spark

Scala

SQL

Big Data

Data Lake

English (C1)

Teamwork

Tools

Control-M

Airflow

Databricks

Azure

AWS

YARN

GitHub

Jenkins

SonarQube

ServiceNow

Job description

Talan Spain is seeking a Big Data Developer to design, develop and evolve data lake solutions for Market Risk use cases in an Agile environment. You will work with Spark, Scala and SQL to build large-scale data pipelines and ensure ingestion, transformation and publication of critical data.

You will collaborate with Agile squads, production support and functional teams to improve performance, reliability and scalability, while supporting incidents and continuous improvement in data operations.