Data Engineering Lead

buscojobs España; Comunidad Valenciana; Presencial; EUR 60.000 - 90.000; Jornada completa

Descripción de la vacante

A leading company in the data engineering field is seeking a Principal Data Engineer to lead the development of scalable data systems. This role involves guiding a team, optimizing hybrid data solutions, and ensuring system reliability. Candidates should have extensive experience in data engineering, particularly with tools like Apache Kafka and AWS, and strong leadership skills.

Formación

• 5+ years in data engineering.
• Production experience with Apache Kafka.
• At least B2 level of English.

Responsabilidades

• Design and implement scalable data pipelines focused on stream processing.
• Lead and mentor a team of engineers, ensuring quality and collaboration.
• Evaluate emerging tools for integration and optimization.

Conocimientos

Data Engineering

Python

Scala

Java

Apache Kafka

AWS

Data Visualization

Leadership

Communication

Educación

Bachelor's Degree

Herramientas

ClickHouse

Redis

Flink

Grafana

RabbitMQ

Superset

Descripción del empleo

We are seeking a Principal Data Engineer to lead the development of scalable, reliable data systems that support real-time analytics and high-throughput data processing. As a Data Engineering Lead, you will architect and guide the implementation of hybrid data solutions that bridge on-premise infrastructure and cloud platforms (preferably AWS).

You will lead a team of skilled data engineers, drive the adoption of best practices in stream processing, and work closely with architecture, product, and software engineering teams to align technical solutions with business needs.

Key Responsibilities

1. Design and lead the implementation of scalable data pipelines with a strong focus on stream processing (Kafka).
2. Guide the migration between cloud and on-premise infrastructure, optimizing for cost and performance.
3. Lead and mentor a team of engineers; define roadmap, ensure quality, and promote collaboration.
4. Implement and maintain high-performance data storage and processing solutions using tools such as ClickHouse, Redis, and Kafka.
5. Ensure system reliability, data consistency, and "at least once" delivery guarantees in streaming architecture.
6. Evaluate emerging tools like Flink, Superset, Cube.js, and Grafana for integration and optimization.
7. Collaborate with cross-functional teams to gather data requirements and align on architecture and goals.
8. Stay informed about trends in data engineering and apply innovation to continually improve infrastructure.

Required Skills

1. 5+ years in data engineering with Python, Scala, or Java.
2. Production experience with Apache Kafka.
3. Strong understanding of relational and analytical databases (e.g., PostgreSQL, MySQL, ClickHouse).
4. Hands-on experience with AWS or other cloud platforms, and on-premise environments.
5. Familiarity with Flink, Redis, RabbitMQ, and data visualization tools (Grafana, Superset) is a plus.
6. Strong communication and leadership skills.
7. Experience in the iGaming industry is a plus.
8. At least B2 level of English.