Senior Data Engineer

Airswift; Greater London; Hybrid; GBP 90,000 - 140,000; Full time

Job summary

Airswift is seeking a Senior Data Engineer to join a high-performing product team delivering enterprise-scale data platforms and data products for critical decision-making in a trading environment.

The role focuses on designing scalable data pipelines with Python, PySpark, Databricks, and SQL Server, while building FastAPI services and REST APIs for data access. Hybrid London-based, with 3 days in office.

Qualifications

• Proven experience building scalable data pipelines and platforms.
• Proficiency in Python, PySpark, and Databricks.
• Experience with CI/CD, DevOps practices, and containerised apps.

Responsibilities

• Design, develop, and maintain scalable data pipelines and data products using Python, PySpark, Databricks, SQL Server, and Azure.
• Build and support FastAPI services and REST APIs that enable data access and integration.
• Collaborate with product owners, architects, and business stakeholders to deliver robust technical solutions.
• Lead development across the full software lifecycle, including design, testing, deployment, monitoring, and support.
• Implement and optimise CI/CD pipelines, automated testing, and release processes using GitOps and modern DevOps practices.
• Develop and support containerised applications using Docker and Kubernetes.
• Drive engineering excellence through code reviews, observability, security, and operational best practices.
• Troubleshoot production issues and continuously improve platform performance and reliability.

Skills

Python

PySpark

Databricks

FastAPI

SQL Server

Docker

GitOps

CI/CD/DevOps

Data pipelines

Tools

Azure

Kubernetes

Job description

Location: London (3 days per week in office)

Our client is seeking a Senior Data Engineer to join a high-performing product team delivering enterprise-scale data platforms and data products that drive critical decision-making across a complex trading environment.

Key Responsibilities:

• Design, develop, and maintain scalable data pipelines and data products using Python, PySpark, Databricks, SQL Server, and Azure.
• Build and support FastAPI services and REST APIs that enable data access and integration.
• Collaborate with product owners, architects, and business stakeholders to deliver robust technical solutions.
• Lead development across the full software lifecycle, including design, testing, deployment, monitoring, and support.
• Implement and optimise CI/CD pipelines, automated testing, and release processes using GitOps and modern DevOps practices.
• Develop and support containerised applications using Docker and Kubernetes.
• Drive engineering excellence through code reviews, observability, security, and operational best practices.
• Troubleshoot production issues and continuously improve platform performance and reliability.

Must-Have Skills:

• Python
• PySpark
• Databricks
• FastAPI
• SQL Server
• Docker
• GitOps
• CI/CD and DevOps practices
• Data platform and data pipeline engineering experience

Nice-to-Have:

• Trading, commodities, or energy sector experience
• Data architecture and data modelling expertise