Data Engineer

Radley James; Greater London; On-site; GBP 180,000 - 220,000; Full time

Job summary

Radley James is partnering with a leading investment firm in London to hire an experienced Data Engineer who can help build and scale a high-performance data platform that supports quantitative investment teams. The role focuses on large-scale data infrastructure, backtesting and research workloads, handling petabyte-scale datasets, and collaborating with quantitative engineers and researchers.

You will design batch and streaming architectures, optimize performance, and improve reliability of

Qualifications

• 5+ years in data-intensive engineering
• Strong SQL and database expertise with large-scale/time-series datasets
• Strong programming in Python, Rust and/or C++
• Experience with Pandas, Polars, Dask or PySpark
• Experience building data platforms, ETL systems, data lakes/warehouses/lakehouse
• Knowledge of Parquet, Arrow or similar formats
• Experience with distributed systems like Kafka and Redis
• Strong understanding of performance optimisation and debugging
• Nice to have: ClickHouse, Snowflake, Prometheus, Grafana or Sentry

Responsibilities

• Building and scaling data infrastructure for backtesting and data-intensive applications
• Developing ingestion and ETL pipelines over petabyte-scale datasets
• Solving data quality, storage, backfills and high-performance data consumption challenges
• Designing batch and streaming data architectures
• Collaborating with quantitative researchers and engineering teams
• Improving scalability, reliability and performance of distributed data systems

Skills

SQL

Python

Rust

C++

Pandas

Polars

Dask

PySpark

Data platforms

ETL pipelines

Parquet/Arrow

Kafka

Redis

Performance optimization

Tools

Kafka

Redis

Job description

Location: London

Compensation: Up to £200,000 base + performance bonus

I’m working with a leading investment firm in London that is looking to hire an experienced Data Engineer to help build and scale a high-performance data platform supporting quantitative investment teams.

This is a highly technical engineering role focused on large-scale data infrastructure, backtesting and research workloads. You’ll be working with petabyte-scale datasets and partnering closely with quantitative engineers and researchers.

What you’ll be working on:

• Building and scaling data infrastructure for backtesting and other data-intensive applications
• Developing ingestion and ETL pipelines operating across petabyte-scale datasets
• Solving challenges around data quality, storage, backfills and high-performance data consumption
• Designing batch and streaming data architectures
• Working closely with quantitative researchers and engineering teams
• Improving the scalability, reliability and performance of distributed data systems

What we’re looking for:

• 5+ years of experience in data-intensive engineering
• Strong SQL and database expertise, particularly with large-scale or time-series datasets
• Strong programming skills in Python, Rust and/or C++
• Experience with tools such as Pandas, Polars, Dask or PySpark
• Experience building data platforms, ETL systems, data lakes, warehouses or lakehouse architectures
• Knowledge of Parquet, Arrow or similar columnar formats
• Experience with distributed systems technologies such as Kafka and Redis
• Strong understanding of performance optimisation and debugging

Nice to have:

• ClickHouse, Snowflake or similar technologies
• Prometheus, Grafana or Sentry

You’ll have the opportunity to work on genuinely large-scale data engineering problems where performance and reliability matter, while building infrastructure used directly by quantitative investment teams.