Data Engineer

Radley James; New York (NY); On-site; USD 175,000 - 250,000; Full time

Job summary

Radley James is partnering with a leading investment firm in New York to hire an experienced Data Engineer. You will build and scale a high-performance data platform for quantitative investment teams, handling petabyte-scale datasets and collaborating with researchers and engineers.

The role focuses on large-scale data infrastructure, backtesting workloads, and designing batch/streaming architectures to improve scalability and reliability across distributed systems.

Qualifications

• 5+ years of experience in data-intensive engineering.
• Strong SQL and database expertise, particularly with large-scale or time-series datasets.
• Strong programming skills in Python, Rust and/or C++.
• Experience with tools such as Pandas, Polars, Dask or PySpark.
• Experience building data platforms, ETL systems, data lakes, warehouses or lakehouse architectures.
• Knowledge of Parquet, Arrow or similar columnar formats.
• Experience with distributed systems technologies such as Kafka and Redis.
• Strong understanding of performance optimisation and debugging.

Responsibilities

• Building and scaling data infrastructure for backtesting and other data-intensive applications.
• Developing ingestion and ETL pipelines operating across petabyte-scale datasets.
• Solving challenges around data quality, storage, backfills and high-performance data consumption.
• Designing batch and streaming data architectures.
• Working closely with quantitative researchers and engineering teams.
• Improving the scalability, reliability and performance of distributed data systems.

Skills

SQL

Python

Rust

C++

Distributed systems

Performance optimization

Tools

Pandas

Polars

Dask

PySpark

Job description

Location: New York, NY

Compensation: Up to $250,000 base + performance bonus

I’m working with a leading investment firm in New York that is looking to hire an experienced Data Engineer to help build and scale a high-performance data platform supporting quantitative investment teams.

This is a highly technical engineering role focused on large-scale data infrastructure, backtesting and research workloads. You’ll be working with petabyte-scale datasets and partnering closely with quantitative engineers and researchers.

What you’ll be working on:

• Building and scaling data infrastructure for backtesting and other data-intensive applications
• Developing ingestion and ETL pipelines operating across petabyte-scale datasets
• Solving challenges around data quality, storage, backfills and high-performance data consumption
• Designing batch and streaming data architectures
• Working closely with quantitative researchers and engineering teams
• Improving the scalability, reliability and performance of distributed data systems

What we’re looking for:

• 5+ years of experience in data-intensive engineering
• Strong SQL and database expertise, particularly with large-scale or time-series datasets
• Strong programming skills in Python, Rust and/or C++
• Experience with tools such as Pandas, Polars, Dask or PySpark
• Experience building data platforms, ETL systems, data lakes, warehouses or lakehouse architectures
• Knowledge of Parquet, Arrow or similar columnar formats
• Experience with distributed systems technologies such as Kafka and Redis
• Strong understanding of performance optimisation and debugging

Nice to have:

• ClickHouse, Snowflake or similar technologies
• Prometheus, Grafana or Sentry

You’ll have the opportunity to work on genuinely large-scale data engineering problems where performance and reliability matter, while building infrastructure used directly by quantitative investment teams.

Compensation: $175,000 to $250,000 base + bonus