Big Data Engineer

Aldia; España; Hybrid; EUR 70,000 - 110,000; Full time

Benefits offered by this job

Hybrid working model

International Health Insurance

Health and Wellbeing Package (350 EUR)

Birthday Day Off

Me Time 1 day off per year

Great career development opportunities

Job summary

GiG in Spain is seeking a Senior Big Data Engineer to own design, build, and support major realtime data projects. You will drive data-driven decisions, review code, promote best practices, and implement robust data pipelines in a hybrid work setting.

You'll work with Spark, Kafka, Nifi, ClickHouse and SQL to process and validate large datasets, design scalable data models, and troubleshoot live systems while exploring new technologies to improve performance.

Qualifications

• Expertise with Spark, Kafka, Nifi and ClickHouse for large-scale data pipelines.
• Strong SQL skills and ability to extract data from complex sources.
• Experience with object-oriented programming in Python and/or Java.
• Understanding of data models, schema design and distributed cloud architectures.

Responsibilities

• Design, create, and maintain ETL processes, data integrations, and pipelines.
• Ensure seamless handling of vast amounts of data and data quality checks.
• Develop high-performance services to process, store, and analyze large datasets.
• Lead data validation, troubleshoot live production systems, and implement new technologies.

Skills

SQL

Python

Java

Cloud/Distributed Systems

Tools

Spark

Kafka

Nifi

ClickHouse

Job description

At GiG we are looking to hire a Senior Big Data Engineer in our Data department.You will assume a pivotal role in taking ownership of the analysis, design, building, support of major realtime big data projects and contribute tothe team's success through code reviews, promoting best practices andimplementing robust solutions across all aspects of big data endeavours. Embracing this vital role, the Senior Big Data Engineer is instrumental in driving data-driven excellence, harnessing the power of information for strategic decision-making, and fostering organisational growth and success.

Key Responsibilities

• Design, create, and maintain ETL processes, data integrations, and data pipelines
• Ensure seamless handling of vast amounts of data.
• Leverage extensive knowledge, the engineer develops high-performance services to efficiently process, store, and analyse large datasetsusing optimised code.
• Aim to maintain data accuracy and reliability, takes the lead in designing and implementing data validation and quality checks tailored to meet specific business needs.
• Troubleshoot live production systems is a regular part of the engineer's responsibilities, ensuring smooth operations and swift issue resolution.
• Demonstrate an innovative spirit, the engineer actively tests andimplements new technologies, staying abreast of the latest advancements relevant to current and future projects.

Requirements

• Strong level of expertise with big data technologies such as Spark, Kafka, Nifi, ClickHouse and Big Data datastores.
• Strong understanding of schema design methodologies and data models
• Expert in extracting large datasets using SQL, familiarity with SQLdatabases
• Experience with object oriented programming skillsideally Python and/or JavaStrong experience with cloud computed and/or distributed systems

Benefits

• Great career development opportunities
• Hybrid working model
• International Health Insurance
• Health and Wellbeing Package (350 EUR per year)
• Birthday Day Off
• Me Time1 day off per year

DepartmentTechnologyEmployment TypeFull TimeWorkplace typeHybrid

Big Data Engineer • Les, Kingdom Of Spain, ES