Senior Engineer, Commodities Technology

Cross Atlantic Partners; Greater London; On-site; GBP 90,000 - 130,000; Full time

Job summary

Cross Atlantic Partners is seeking a Senior Data Engineer to join its Commodities Technology team in London. You will design and build data platforms powering real-time trading decisions, working directly with traders and quantitative researchers.

You will own production systems in a fast-paced, low-latency environment, developing cloud-native, containerised data platforms and Python services. This role emphasizes on-call responsibilities and high-impact data engineering solutions.

Qualifications

• 5+ years' experience in Data Engineering with ownership of production systems.
• Expert Python development for production services and bespoke data pipelines.
• Strong SQL skills with data modelling and optimisation.
• Experience building real-time or near real-time data pipelines.
• Commercial experience with Kafka, Kubernetes and Docker.
• Strong understanding of Git, code reviews, CI/CD and testing practices.
• Exposure to Apache Airflow is beneficial.

Responsibilities

• Design, build and maintain high-performance, low-latency data pipelines.
• Develop production-grade Python services and data engineering solutions.
• Own production systems, on-call rota and incident resolution.
• Work with front-office users to deliver reliable data.
• Build and improve cloud-native, containerised data platforms.
• Drive engineering best practices through code reviews, testing and CI/CD.

Skills

Python

SQL

Kafka

Kubernetes

Docker

Git

CI/CD

Airflow

Tools

Apache Airflow

Kafka

Kubernetes

Docker

Git

Job description

Location: London

A leading global investment firm is looking for a Senior Data Engineer to join its Commodities Technology team. Sitting at the intersection of front-office trading and enterprise data engineering, you\'ll design and build the data platforms that power real-time trading decisions.

This is a highly visible role where you\'ll work directly with traders, portfolio managers and quantitative researchers, owning business-critical data systems in a fast-paced, low-latency environment.

What you\'ll be doing

• Design, build and maintain high-performance, low-latency data pipelines.
• Develop production-grade Python services and data engineering solutions.
• Own critical production systems, participating in an on-call rota and resolving live incidents.
• Work closely with front-office users to deliver reliable, high-quality data.
• Build and improve cloud-native, containerised data platforms.
• Drive engineering best practices through code reviews, testing and CI/CD.

What we\'re looking for

• 5+ years\' experience in Data Engineering with ownership of production systems.
• Expert Python development for production services and bespoke data pipelines.
• Strong SQL skills with experience in complex data modelling and optimisation.
• Experience building real-time or near real-time data pipelines.
• Commercial experience with Kafka, Kubernetes and Docker.
• Strong understanding of Git, code reviews, CI/CD and testing practices.
• Experience supporting business-critical production systems.
• Exposure to Apache Airflow is beneficial.
• Commodities or energy market experience (power, gas, oil, agriculture or shipping) is advantageous but not essential.

Why apply?

• Work directly alongside traders and quantitative researchers.
• Build systems that have a direct impact on trading performance.
• Join a collaborative engineering team focused on modern cloud-native technologies.
• Take ownership of business-critical platforms in a highly technical environment.
• Excellent compensation, bonus potential and long-term career progression.

If you\'re an experienced Data Engineer looking to move into a high-performance front-office environment where your work directly influences trading, we\'d love to hear from you.