Senior Data Engineer

Quantum; Greater London; Hybrid; GBP 85,000 - 120,000; Full time

Benefits offered by this job

Competitive Base Salary

Company Bonus Scheme

Hybrid Working (3 days in office)

Private Health Care (Bupa)

Travel Insurance

Market Leading Training Programme

Recognition & Reward Scheme

Annual Company Conference

Regular Happy Hour / Team Lunches

Free Coffee, Drinks & Snacks

Job summary

A leading digital comparison service in the UK is seeking a Senior Data Engineer to design and build scalable real-time data pipelines. The role involves integrating data sources, ensuring efficient data flow, and collaborating with teams to support growing data needs. Ideal candidates will have extensive experience in data engineering, cloud platforms, and programming skills in languages such as Python or Java. This position also offers a competitive salary, hybrid working model, and a range of benefits including private health care and training programs.

Qualifications

• Hands-on experience in data engineering with a proven track record.
• Expertise in developing data backbones using distributed streaming platforms.
• Experience with cloud platforms for real-time data ingestion.
• Programming skills in Python, Java, Scala, or similar.
• Proficiency in SQL, NoSQL, and time-series databases.
• Strong understanding of orchestration tools.

Responsibilities

• Design, develop, and maintain scalable real-time data pipelines.
• Integrate various data sources and maintain seamless data flow.
• Build efficient and fault-tolerant data ingestion processes.
• Monitor and improve the performance of data pipelines.
• Collaborate with cross-functional teams to align infrastructure with business goals.

Skills

Data engineering

Real-time data pipelines

Cloud platforms (AWS, GCP, Azure)

Programming skills (Python, Java, Scala)

Database technologies (SQL, NoSQL)

Data pipeline orchestration tools

Team collaboration

Fast-paced environment adaptability

Excellent communication skills

Tools

Kafka

Spark Streaming

Apache Airflow

Kubernetes

Flink

Job description

Overview

Our Company At Quantum, we connect global brands with high-intent consumers. We turn intent into action and complexity into clarity.

Our Mission is clear, to help smarter, faster buying decisions. Our vision, to be the world’s most used digital comparison service, delivering value and simplicity to millions.

We drive growth through data-driven affiliate marketing built for performance from day one. Every campaign is measurable, optimised in real time, and designed to scale.

With in-house tech, full-stack analytics, and a compliance-first mindset, we give our partners an edge. We drive confident decisions and growth in regulated and high-growth sectors. We’ve helped ambitious brands grow faster, smarter, and globally.

The Role

Our Technology team is growing! We’re looking for a highly skilled and experienced Senior Data Engineer to design and build a robust, scalable, and high-performance real-time data backbone.

In this role you will lead the design, development, and implementation of a real-time data backbone. This position focuses on building and maintaining scalable, high-performance data pipelines that support real-time data processing and analytics. You will work closely with various teams to ensure the infrastructure meets the business\'s growing data needs while leveraging modern technologies for efficient data ingestion, storage, and analysis.

Key Responsibilities

• Design, develop, and maintain scalable real-time data pipelines and infrastructure.
• Integrate various data sources and ensure seamless real-time data flow across the organisation.
• Build efficient, fault-tolerant, and highly available data ingestion processes.
• Monitor and improve data pipeline performance and scalability for low-latency and high-throughput.
• Work closely with cross-functional teams, including software engineers, data scientists, and business stakeholders, to ensure data infrastructure aligns with business objectives.

About You

• Hands-on experience in data engineering with a proven track record of building and managing real-time data pipelines across a track record of multiple initiatives.
• Expertise in developing data backbones using distributed streaming platforms (Kafka, Spark Streaming, Flink, etc.).
• Experience working with cloud platforms such as AWS, GCP, or Azure for real-time data ingestion and storage.
• Programming skills in Python, Java, Scala, or a similar language.
• Proficiency in database technologies (SQL, NoSQL, time-series databases) and data modelling.
• Strong understanding of data pipeline orchestration tools (e.g., Apache Airflow, Kubernetes).
• You thrive when working as part of a team
• Comfortable in a fast-paced environment
• Have excellent written and verbal English skills
• Last but not least, you’ll have no ego!

What You’ll Get

• Competitive Base Salary
• Company Bonus Scheme
• Hybrid Working (3 days in office)
• Private Health Care (Bupa)
• Travel Insurance
• Market Leading Training Programme
• Recognition & Reward Scheme
• Annual Company Conference (previous destinations Vienna, Bologna, Dubrovnik and Thessaloniki)
• Regular Happy Hour / Team Lunches
• Free Coffee, Drinks & Snacks

What’s the next step?

Our hiring process ensures we\'re recruiting the right people for the role. We ensure that people are as suitable for us as we are for them.

If you like the sound of what we\'re all about at Quantum and want to join a team where you can make an impact, please apply or contact us at [e-mail].