Senior Data Engineer

Stream Hatchet; Valencia; Presencial; EUR 45.000 - 65.000; Jornada completa

Descripción de la vacante

A leading data analytics company in the esports and gaming streaming industries is seeking a Senior Data Engineer to drive the migration of data systems to a scalable cloud-native architecture. This role involves designing data pipelines and optimizing workflows to handle social media metrics from millions of creators, requiring expertise in Python, cloud services, and robust data engineering practices.

Formación

• 3+ years of experience in data engineering
• Proficiency in Python and AWS services
• Understanding of ETL processes and data architecture patterns

Responsabilidades

• Lead migration to a Python-based cloud-native architecture
• Design and maintain scalable data ingestion pipelines
• Ensure data quality and reliability across ingestion processes

Conocimientos

Data pipeline development

Cloud-native development

Data aggregation

Data quality

SQL optimization

Elasticsearch

Python

Educación

BS degree in Computer Science

Herramientas

AWS

SQL databases

NoSQL databases

Ruby on Rails

Descripción del empleo

Stream Hatchet is a leading data analytics company in the esports and gaming streaming industries. We leverage data from social platforms like Twitter, Instagram, TikTok, and live streaming platforms to provide actionable insights for businesses focused on go-to-market strategies, content initiatives, and sponsorship measurement.

As a Senior Data Engineer, you will play a key role in the migration of our current data ingestion and aggregation system from a Ruby-on-Rails monolith to a more scalable, cloud-native architecture. You will work closely with our engineering team to design and implement efficient data pipelines that can scale to ingest, process, and aggregate social media metrics from millions of creators daily.

If you thrive in fast-paced environments, enjoy solving complex technical challenges, and have a passion for building scalable cloud architectures, this is the role for you!

Responsibilities

• Lead the migration from a Ruby-on-Rails system to a Python-based, cloud-native architecture.
• Design, build, and maintain scalable data ingestion and aggregation pipelines using cloud services.
• Optimize existing data workflows to improve reliability, scalability, and performance.
• Collaborate with cross-functional teams to ensure seamless integration of new data sources and the delivery of data insights.
• Ensure data quality, accuracy, and reliability across the entire ingestion and aggregation process.
• Evaluate and implement best practices for monitoring, alerting, and maintaining production-level data systems.

Requirements

• BS degree in Computer Science, Engineering, or a related technical field.
• 3+ years of experience in data engineering, including data pipeline development, aggregation, and ingestion.
• Proficiency in Python and experience with cloud-native development using AWS services.
• Proficiency with SQL databases and query optimization. Familiarity with NoSQL databases is a plus.
• Understanding of data architecture patterns, ETL processes, and distributed data systems.
• Familiarity with Ruby on Rails is a plus.
• Experience with Elasticsearch for building search and analytics solutions.
• Excellent communication skills in English.

J-18808-Ljbffr