Senior Data Engineer, Real-Time Commodities Tech

Cross Atlantic Partners; Greater London; On-site; GBP 90,000 - 130,000; Full time

Job summary

Cross Atlantic Partners is seeking a Senior Data Engineer to join its Commodities Technology team in London. You will design and build data platforms powering real-time trading decisions, working directly with traders and quantitative researchers.

You will own production systems in a fast-paced, low-latency environment, developing cloud-native, containerised data platforms and Python services. This role emphasizes on-call responsibilities and high-impact data engineering solutions.

Qualifications

• 5+ years' experience in Data Engineering with ownership of production systems.
• Expert Python development for production services and bespoke data pipelines.
• Strong SQL skills with data modelling and optimisation.
• Experience building real-time or near real-time data pipelines.
• Commercial experience with Kafka, Kubernetes and Docker.
• Strong understanding of Git, code reviews, CI/CD and testing practices.
• Exposure to Apache Airflow is beneficial.

Responsibilities

• Design, build and maintain high-performance, low-latency data pipelines.
• Develop production-grade Python services and data engineering solutions.
• Own production systems, on-call rota and incident resolution.
• Work with front-office users to deliver reliable data.
• Build and improve cloud-native, containerised data platforms.
• Drive engineering best practices through code reviews, testing and CI/CD.

Skills

Python

SQL

Kafka

Kubernetes

Docker

Git

CI/CD

Airflow

Tools

Apache Airflow

Kafka

Kubernetes

Docker

Git

Job description

Cross Atlantic Partners is seeking a Senior Data Engineer to join its Commodities Technology team in London. You will design and build data platforms powering real-time trading decisions, working directly with traders and quantitative researchers.

You will own production systems in a fast-paced, low-latency environment, developing cloud-native, containerised data platforms and Python services. This role emphasizes on-call responsibilities and high-impact data engineering solutions.