Senior Data Engineer

Mustard Systems Ltd.; Greater London; Hybrid; GBP 70,000 - 90,000; Full time

Benefits offered by this job

Hybrid working environment

Competitive salary

Health insurance

Enhanced pension match

Sabbatical leave

33 days annual leave

Job summary

A leading technology company is seeking a Senior Data Engineer to join their Horse Racing team in London. The ideal candidate has over 5 years of experience and expertise in Python and SQL. Responsibilities include building data infrastructure and ensuring data quality. This innovative environment values creativity and experimentation, offering a hybrid working model and competitive benefits including health insurance and 33 days of annual leave.

Qualifications

• 5+ years of experience in data engineering.
• Deep experience in Python, SQL, and data modelling.
• Hands-on experience with streaming technologies like Kafka.

Responsibilities

• Build and maintain data infrastructure for trading operations.
• Design and implement streaming data architectures.
• Ensure data quality and reliability through testing and validation.

Skills

Python

SQL

Data modelling

Communication skills

Self-starter attitude

Decision-making

Education

Degree in Computer Science or a numerical subject

Tools

Snowflake

Dagster

DBT

Kafka

ClickHouse

AWS

Job description

Are you an engineer who thrives in a fast-paced, experimental environment? Do you relish the challenge of building complex data systems, testing ideas, and learning from failure as much as success? Mustard Systems is seeking a Senior Data Engineer to join our talented and high-growth Horse Racing team, where you'll collaborate with a unique blend of mathematicians, statisticians, international chess masters, and Countdown Octo-Champs to tackle some of the most complex and exciting problems in sports prediction.

What You Won't Have To Do In This Role

• Be bogged down by red tape or excessive bureaucracy
• Check in repeatedly or wait for permission to try new ideas
• Aim for perfect code or endless code reviews

The Horse Racing team specialises in predicting the outcomes of Horse Racing around the world, building in-house sophisticated trading systems and predictive models.

In this role, you'll work alongside some of the sharpest minds in the industry, in a culture that values creativity, experimentation, and diversity of thought. If you're ready to make an impact by innovating at the cutting edge of sports prediction, we'd love to hear from you.

This isn't a role for a by-the-book engineer. Instead, it's perfect for someone who enjoys exploring uncharted territory, using their technical expertise to experiment, innovate, and deliver rapid results.

What You'll Do

• Build and maintain the data infrastructure that powers our trading operations. This includes:
• Building and maintaining data pipelines that ingest data from multiple external providers, APIs, and real-time feeds
• Designing and implementing streaming data architectures to support low-latency trading requirements
• Optimising our data warehouse for both cost and performance as data volumes grow
• Ensuring data quality and reliability through testing, validation, and CI/CD best practices
• Improving observability, monitoring, and alerting around data freshness, volume anomalies, and pipeline health
• Partnering with quantitative analysts and traders to understand data requirements and deliver robust solutions
• Mentoring other team members on technical decisions and best practices

Requirements

• 5+ years of experience in data engineering, with deep experience in Python, SQL, and data modelling
• Experience with cloud data warehouses, orchestration frameworks, and transformation tools
• Hands-on experience with streaming technologies (Kafka or similar)
• Experience integrating data from external APIs and third-party providers
• Experience working in environments where the speed of development is prioritised over formal processes
• A self-starter attitude, with the confidence to take ownership of projects and experiment with new ideas
• Strong decision-making abilities, with a knack for making thoughtful trade-offs balancing speed, quality, and maintainability
• Excellent communication skills - able to discuss technical concepts clearly with both technical and non-technical colleagues
• A degree in Computer Science or a numerical subject from a top university

Our Tech Stack

• Snowflake for data warehousing
• Dagster for orchestration
• Python 3.12+
• DBT for transformations
• Kafka for streaming
• ClickHouse for real-time analytics
• On-Prem (Linux) + AWS

Your Portfolio or Personal Projects

We're especially keen to see what you've built outside of your day job. Whether it's a passion project, an experimental tool, or something a little quirky, we'd love to hear about it. These projects often tell us more about your creativity and approach to problem-solving than a standard CV ever could.

Benefits

• Hybrid working environment. We're in the office every Monday, Tuesday and Thursday, and work from home every Wednesday and Friday
• Work on cutting-edge systems in a competitive and innovative field
• Collaborate with a smart, driven team, where your contributions directly impact business performance
• Opportunity to drive the company's technical direction and double its revenue in the next three years

Comprehensive benefits, including:

• Competitive salary and significant bonus potential
• Enhanced pension match with salary sacrifice option
• Health insurance and life assurance
• Sabbatical leave after five years
• 33 days of annual leave (including bank holidays)