Data Engineer

DigiGamma; Haarlem; Hybrid; EUR 70,000 - 95,000; Full time

Benefits offered by this job

Remote work once a week

Continuous learning budget

Free coffee/tea/fruits

Job summary

DigiGamma is seeking a Data Engineer to design, develop, and operate large-scale data systems in a fast-growing Dutch e-commerce environment. You will focus on real-time indexing pipelines, streaming analytics, and distributed ML infrastructure, collaborating with engineers, PMs, and architects to deliver robust solutions.

Ideal candidates have 4+ years in object-oriented programming (Java/Scala), 3+ years building data pipelines with Spark/Kafka/Hadoop ecosystem, and strong CS fundamentals.

Qualifications

• Bachelor’s degree in Computer Science or related field.
• 4+ years of object-oriented programming experience in Java or Scala.
• 3+ years building large-scale data pipelines using Spark/Kafka/Cassandra/Hadoop/Hive/Presto/Airflow.
• 3+ years of experience in systems design, algorithms, and distributed systems.
• 3+ years of scripting languages (e.g. Python), and SQL.

Responsibilities

• Design, develop and operate large-scale data systems focusing on scalability, latency, and fault tolerance.
• Create and operate web analytics solutions handling millions of daily clickstream events.
• Own end-to-end life cycle of business data solutions from requirements to production.
• Troubleshoot business and production issues and optimize for speed and cost.

Skills

Distributed systems

Cloud infrastructure

Communication skills

Education

Bachelor's degree in Computer Science or related field

Tools

Spark

Kafka

Cassandra

Hadoop

Hive

Presto

Airflow

Docker

Kubernetes

OpenStack

AWS

GCP

Job description

This is an exclusive opportunity to work as a Data Engineer in one of the fastest growing e-commerce start-ups based in the Netherlands. We are looking to expand our technical team with a Data Engineer to invest more in data driven technologies.

Key Qualifications

• Design, develop, and implement large-scale distributed systems that process large volume of data; focusing on scalability, latency, and fault-tolerance in every system built.
• Design, develop, and operate web analytics solutions processing millions on daily clickstream events.
• Create systems that orchestrate and execute complex workflows in big-data pipelines.
• Evaluate and fine tune systems for speed, robustness, and cost efficiency.
• Create datasets, tools, and services supporting big data, search and machine learning operations.
• Own full life-cycle of business solutions from requirements definition to launching in production.
• Troubleshoot business and production issues.
• Own multiple systems across the search big data platforms, and work with engineers, program managers, and engineering leaders to identify opportunities for business impact.
• Participate in setting a vision and objectives for the team in alignment with business and market needs.

Description

Description
You’ll be responsible for design, development and operations of large-scale data systems within the DigigGamma ecosystem. You will be focusing on real-time indexing pipelines, streaming analytics, distributed machine learning infrastructure and other tasks as part of Search BigData team. You’ll interact with the engineers, product managers and architects to provide scalable robust technical solutions.

Qualifications

• Bachelor’s degree in Computer Science or related technical field.
• 4+ years of object-oriented programming experience in Java or Scala.
• 3+ years of experience in building of large scale data pipelines using big data technologies (i.e. Spark/Kafka/Cassandra/Hadoop/Hive/Presto/Airflow).
• 3+ years of experience in systems design, algorithms, and distributed systems.
• 3+ years of experience in scripting languages (e.g. Python), and SQL.

Skills

• You’ll need to have:
• Large scale distributed systems experience, including scalability and fault tolerance.
• Exposure to infrastructure management tech (Docker, Kubernetes)
• Exposure to cloud infrastructure, such as Open Stack, Azure, GCP, or AWS
• A continuous drive to explore, improve, enhance, automate and optimize systems and tools.
• Strong computer science fundamentals in data structures and algorithms
• Exposure to information retrieval, statistics and machine learning.
• Excellent oral and written communication skills.

What we offer:

• Remote work once a week
• Positive spirit within a top on the line international team
• Share the glory of disrupting the fintech landscape
• Challenging projects and freedom to experiment
• Continuous learning: free English classes, IT certifications, technical events, budget for book or online training sites
• Standard perks: free coffee / tea/ fruits, no dress code