Sr. Data Engineer

The Kynetec Group; Islas Baleares; Presencial; EUR 50.000 - 75.000; Jornada completa

Descripción de la vacante

A leading company in data analytics seeks a Senior Data Engineer to optimize and maintain complex data pipelines. The ideal candidate will drive data quality and security while mentoring junior engineers, focusing on advanced analytics within a Databricks environment. Applicants should possess robust experience in data engineering and tools like SQL and PySpark.

Formación

• 5+ years of experience in data engineering and database systems.
• Strong experience with cloud platforms like AWS and Azure.
• Excellent communication and collaboration skills.

Responsabilidades

• Build and maintain data processing systems and pipelines.
• Collaborate with teams to ensure data quality and accessibility.
• Implement monitoring to ensure data pipeline performance.

Conocimientos

SQL

Python

PySpark

ETL processes

Data quality

Collaboration

Educación

Bachelor's degree in Computer Science, Engineering, or a related field

Master's degree

Herramientas

AWS

Azure

Databricks

Azure Data Factory

Kedro framework

Relational databases

NoSQL databases

Descripción del empleo

• Create and maintain large-scale data processing systems and infrastructure.
• Build robust, performant, and scalable data pipelines to ingest, transform, and store complex data from multiple sources.
• Collaborate with data science and software engineering team members to provide required data in an accessible, timely, and accurate manner.
• Optimize and refine processes, algorithms, and systems to enhance data quality and reliability.
• Ensure data privacy and security compliance.
• Implement monitoring, logging, and alert systems to ensure data pipeline health and performance.
• Collaborate with infrastructure and IT teams to ensure optimal data storage and retrieval mechanisms.
• Drive the optimization, testing, and tooling to improve data quality.
• Mentor junior data engineers, imparting knowledge and promoting best practices.
• Stay updated with emerging technologies and introduce them as needed to improve the data engineering ecosystem.

Requirements :

• Bachelor's degree in Computer Science, Engineering, or a related field. Master's degree is a plus.
• 5+ years of experience in data engineering, ETL processes, and database systems.
• Proficient in SQL, Python, PySpark, and experience with big data platforms.
• Experience creating data pipelines and working with workflow management tools.
• Strong experience with relational and NoSQL databases.
• Working experience in cloud platforms like AWS and Azure.
• Excellent communication and collaboration skills.
• Experience with Databricks platform, Azure Data Factory, Kedro framework.
• Experience with Machine Learning processes.
• Relevant Industry Certification.

Role Description

The Senior Data Engineer will play a pivotal role in building, optimizing, and maintaining our data pipeline architecture, ensuring data quality and accessibility for cross-functional teams. This role will be a key member of our Advanced Analytics team and will focus on building and supporting our data pipelines in a Databricks environment.

J-18808-Ljbffr