Remote Big Data Engineer — Spark, Scala & SQL

Talan; Santiago; Remote; CLP 15,000,000 - 32,000,000; Full time

Benefits offered by this job

Private medical insurance

Life insurance

Lunch and transport cards

Online language classes

Smart Office Pack

100% remote working

Job summary

Talan is seeking a Big Data Developer to join an Agile environment and work on the design, development and evolution of data lake solutions for Market Risk use cases.

You will work with Spark, Scala and SQL to develop large-scale data processing pipelines and contribute to ingestion, transformation, validation and publication of critical data. The role blends hands-on development with collaboration across squads and production support teams.

Qualifications

• 2+ years of professional experience in Big Data, Data Lakes or cloud-based data platforms.
• Strong hands-on experience with Apache Spark, Scala and SQL for large-scale data processing.
• Good understanding of Data Lake architecture, data quality controls, data lineage and production-grade data pipelines.
• Strong analytical and problem-solving skills.
• Excellent communication and teamwork skills, with the ability to collaborate effectively with technical and functional stakeholders.
• Adaptability and a proactive approach to solving technical challenges.
• Spanish is considered a plus.

Responsibilities

• Design, develop and maintain data lake pipelines using Apache Spark, Scala and SQL.
• Build and maintain scalable data processing solutions for large volumes of data.
• Orchestrate batch workflows and dependencies using Control-M and Airflow.
• Implement data ingestion, transformation, validation and publication processes for Market Risk use cases.
• Analyse and optimise the performance, reliability and scalability of Spark-based workloads.
• Analyse requirements, identify gaps and propose effective solutions.
• Contribute to delivery planning and collaborate with different teams to ensure successful implementation.
• Support production incidents and perform root-cause analysis.
• Work with Agile squads, functional teams and production support teams to deliver high-quality solutions.
• Follow software engineering best practices to ensure robust, maintainable production-ready solutions.

Skills

Big Data

Spark

Scala

SQL

Data Lakes

English (C1)

Teamwork

Tools

Control-M

Airflow

Databricks

Azure

AWS

YARN

GitHub

Jenkins

SonarQube

ServiceNow

Job description

Talan is seeking a Big Data Developer to join an Agile environment and work on the design, development and evolution of data lake solutions for Market Risk use cases.

You will work with Spark, Scala and SQL to develop large-scale data processing pipelines and contribute to ingestion, transformation, validation and publication of critical data. The role blends hands-on development with collaboration across squads and production support teams.