Sr. Data Engineer

The Kynetec Group; Sevilla; Presencial; EUR 50.000 - 90.000; Jornada completa

Descripción de la vacante

Join a forward-thinking company as a Senior Data Engineer, where you will play a pivotal role in building and optimizing data pipelines in a Databricks environment. This exciting position involves collaborating with cross-functional teams to ensure data quality and accessibility while mentoring junior engineers. You will have the opportunity to work with cutting-edge technologies and drive improvements in data processing systems. If you're passionate about data engineering and eager to make a significant impact, this role is perfect for you.

Formación

• 5+ years of experience in data engineering and ETL processes.
• Proficient in SQL and Python, with experience in big data platforms.

Responsabilidades

• Create and maintain large-scale data processing systems and infrastructure.
• Build robust data pipelines to ingest, transform, and store complex data.

Conocimientos

Data Engineering

ETL Processes

SQL

Python

PySpark

Big Data Platforms

Data Pipeline Creation

Cloud Platforms (AWS, Azure)

Machine Learning

Communication Skills

Educación

Bachelor's degree in Computer Science

Master's degree in related field

Herramientas

Databricks

Azure Data Factory

Kedro Framework

Descripción del empleo

• Create and maintain large-scale data processing systems and infrastructure.
• Build robust, performant, and scalable data pipelines to ingest, transform, and store complex data from multiple sources.
• Collaborate with data science and software engineering team members to provide required data in an accessible, timely, and accurate manner.
• Optimize and refine processes, algorithms, and systems to enhance data quality and reliability.
• Ensure data privacy and security compliance.
• Implement monitoring, logging, and alert systems to ensure data pipeline health and performance.
• Collaborate with infrastructure and IT teams to ensure optimal data storage and retrieval mechanisms.
• Drive the optimization, testing, and tooling to improve data quality.
• Mentor junior data engineers, imparting knowledge and promoting best practices.
• Stay updated with emerging technologies and introduce them as needed to improve the data engineering ecosystem.

Requirements :

• Bachelor's degree in Computer Science, Engineering, or a related field. Master's degree is a plus.
• 5+ years of experience in data engineering, ETL processes, and database systems.
• Proficient in SQL, Python, PySpark, and experience with big data platforms.
• Experience creating data pipelines and working with workflow management tools.
• Strong experience with relational and NoSQL databases.
• Working experience in cloud platforms like AWS and Azure.
• Excellent communication and collaboration skills.
• Experience with Databricks platform, Azure Data Factory, Kedro framework.
• Experience with Machine Learning processes.
• Relevant Industry Certification.

Role Description

The Senior Data Engineer will play a pivotal role in building, optimizing, and maintaining our data pipeline architecture, ensuring data quality and accessibility for cross-functional teams. This role will be a key member of our Advanced Analytics team and will focus on building and supporting our data pipelines in a Databricks environment.