Senior Big Data Engineer

Tecnalis-Solution-Providers; Madrid; Híbrido; EUR 90.000 - 120.000; Jornada completa

Ventajas ofrecidas por este puesto de trabajo

Hybrid working model

International Health Insurance

Health and Wellbeing Package

Birthday Day Off

Me Time - 1 day off per year

Great career development opportunities

Descripción de la vacante

GiG is seeking a Senior Big Data Engineer to own design, development, and support of major realtime data projects within the Data department. You will help drive data-driven decision making and contribute to robust, scalable data solutions.

The role emphasizes ownership of ETL pipelines, data quality, and optimization of large-scale data processing in a hybrid working environment in Madrid.

Formación

• Strong expertise with Spark, Kafka, NiFi, ClickHouse and Big Data datastores.
• Strong understanding of schema design and data models.
• Proficiency in SQL and experience with SQL databases.
• Experience with Python and/or Java.
• Experience with cloud computing and distributed systems.

Responsabilidades

• Design, create, and maintain ETL processes, data integrations, and data pipelines
• Ensure seamless handling of vast amounts of data.
• Develop high-performance services to process, store, and analyze large datasets with optimized code
• Lead data validation and quality checks tailored to business needs
• Troubleshoot live production systems and ensure smooth operations
• Test and implement new technologies, staying updated with advancements in big data

Conocimientos

SQL

Data modeling

Data pipelines

Python

Java

Herramientas

Spark

Kafka

NiFi

ClickHouse

Big Data Datastores

Cloud platforms

Descripción del empleo

Senior Big Data Engineer

Department: Technology

Employment Type: Full Time

Location: Madrid

Reporting To: Jake Montebello

Description

At GiG we are looking to hire a Senior Big Data Engineer in our Data department. You will assume a pivotal role in taking ownership of the analysis, design, building, support of major realtime big data projects and contribute to the team's success through code reviews, promoting best practices and implementing robust solutions across all aspects of big data endeavours.

Embracing this vital role, the Senior Big Data Engineer is instrumental in driving data-driven excellence, harnessing the power of information for strategic decision-making, and fostering organisational growth and success.

Key Responsibilities

• Design, create, and maintain ETL processes, data integrations, and data pipelines
• Ensure seamless handling of vast amounts of data.
• Leverage extensive knowledge, the engineer develops high-performance services to efficiently process, store, and analyse large datasets
  using optimised code.
• Aim to maintain data accuracy and reliability, takes the lead in designing and implementing data validation and quality checks tailored to meet specific business needs.
• Troubleshoot live production systems is a regular part of the engineer's responsibilities, ensuring smooth operations and swift issue resolution.
• Demonstrate an innovative spirit, the engineer actively tests and
  implements new technologies, staying abreast of the latest advancements relevant to current and future projects.

Requirements

• Strong level of expertise with big data technologies such as Spark, Kafka, Nifi, ClickHouse and Big Data datastores.
• Strong understanding of schema design methodologies and data models
• Expert in extracting large datasets using SQL, familiarity with SQL
• databases
• Experience with object oriented programming skills - ideally Python and/or Java
• Strong experience with cloud computed and/or distributed systems

Benefits

• Great career development opportunities
• Hybrid working model
• International Health Insurance
• Health and Wellbeing Package (350 EUR per year)
• Birthday Day Off
• Me Time - 1 day off per year