Big Data Engineer

Gaming Innovation Group; London; Hybrid; GBP 60,000 - 80,000; Full time

Job summary

A leading technology company is seeking a Big Data Engineer to support their cutting-edge data platform using Java and Apache technologies. You will work closely with an agile team, enhancing the platform and engaging in data migration projects. Exceptional collaboration and programming skills are required. The position offers a 100% remote or hybrid working model along with competitive benefits including health insurance and career development opportunities.

Benefits

Great career development opportunities

100% remote or hybrid working model

International Health Insurance

Health and Wellbeing Package

Birthday Day Off

1 day off - Me Time

Free office lunches

Qualifications

• Strong experience in object-oriented programming using Java.
• Familiar with real-time data processing and ETL methods.
• Ability to work collaboratively within an agile team.

Responsibilities

• Participate in all agile scrum meetings.
• Maintain the data platform through daily reconciliations.
• Implement bug fixes and enhancements to the data platform.
• Take ownership of complex data migration projects.

Skills

Object oriented programming (Java)

Data modelling

ETL processes

Apache Spark

CI/CD

Docker

Version control systems

Team collaboration

Education

Bachelor's Degree in Computer Science

Tools

Apache NiFi

ClickHouse

Job description

Overview

Big Data Engineer – Department: Technology – Employment Type: Full Time – Location: United Kingdom

As part of the Big Data Team, you will be using cutting edge technologies to work on collating data across different sources within GiG. Sources are all in real time, and you will be supporting an event driven micro service lambda architecture. Your focus is to work with the current agile team and contribute to the platform's objectives and continue being an enabler to many other data technology stakeholders such as the Business Intelligence, Data Science and Quality Assurance teams. You will be up for a challenge, to continue maintaining and adding new features to a platform that ingests up to 5000 messages per second across multiple products such as Lottery, Sports and Casino. Reporting to the Lead Big Data Engineer and supported by the Big Data Architect to follow best practices, you'll be guided into delivering world class product increments that support our Tier 1 Platform.

Key Responsibilities

• Participate in all agile scrum meetings such as daily standups, refinement sessions, retro sessions, etc
• Maintaining the data platform by means of daily reconciliations, data reloads, tackling support tickets, etc.
• Being able to come up with new ideas to improve existing products and services.
• Implementation of bug fixes and enhancements within the data platform.
• Performing code reviews for other engineers.
• Taking ownership of releases where necessary
• Being able to communicate with the respective stakeholders, making sure that all information is transmitted correctly and to the right audience.
• Take ownership of complex, large-scale initiatives, including data migration projects, ensuring successful planning, execution, and delivery.

Qualifications

You're really awesome at:

• Object oriented programming (Java)
• Data modelling using any database technologies
• ETL processes (ETLs are oldschool, we transfer in memory now) and experience with Apache Spark or Apache NiFi
• Applied understanding of CI\CD in change management
• Dockerised applications
• Used distributed version control systems
• Excellent team player
• Meticulous and passionate about your work

You're also good at:

• Functional oriented programming (experience with Python, Scala)
• Analytical mindset
• Low latency databases such as ClickHouse
• Hold a Bachelor\'s Degree in Computer Science or equivalent
• Can collaborate in scoping exercises
• Mentoring skills
• Have worked with both Linux and Windows environments
• Unix/Linux

Job Benefits

• Great career development opportunities
• 100% remote or hybrid working model
• International Health Insurance
• Health and Wellbeing Package (350 EUR per year)
• Birthday Day Off
• Me Time - 1 day off per year
• Lunches in the office for free