Data Engineer - Defence

iO Associates; Greater London; Hybrid; GBP 89,000 - 148,000; Full time

Job summary

London-based Data Engineer is sought on a contract basis to support a MoD-owned data platform. The role focuses on building and maintaining secure data pipelines using Python, Azure and Kubernetes, with Apache Airflow and Apache Spark integrated into the stack.

You will work within a government-aligned, secure environment, ensuring data remains hosted on MoD infrastructure while collaborating with delivery teams to troubleshoot and improve pipelines.

Qualifications

• Strong commercial experience as a Data Engineer.
• Experience working with Azure.
• Experience with Kubernetes.
• Hands-on experience with Apache Airflow and Apache Spark.
• Experience building and maintaining data pipelines.
• Experience working within government, defence or other secure environments.
• Strong understanding of data engineering principles and best practices.

Responsibilities

• Build and maintain new and existing data pipelines.
• Develop data solutions using Python, Azure and Kubernetes.
• Work with Apache Airflow and Apache Spark.
• Support the ongoing development and improvement of the data platform.
• Ensure data remains securely hosted on MoD-owned infrastructure.
• Collaborate with wider technical and delivery teams.
• Troubleshoot and resolve technical issues across existing data pipelines.

Skills

Data engineering fundamentals

Secure environments experience

Data pipelines

Tools

Python

Azure

Kubernetes

Apache Airflow

Apache Spark

Job description

Contract

Location: London – 4 days per week on-site, 1 day remote

Rate: Up to £800 per day Inside IR35

Clearance: Active DV Clearance

I’m currently recruiting for a Data Engineer to join a long-running programme within Central Government, supporting the development and maintenance of secure data platforms and pipelines across MoD-owned infrastructure.

You’ll work as part of a technical team building and supporting data pipelines using Python, Azure and Kubernetes, alongside technologies including Apache Airflow and Apache Spark.

Key Responsibilities

• Build and maintain new and existing data pipelines
• Develop data solutions using Python, Azure and Kubernetes
• Work with Apache Airflow and Apache Spark
• Support the ongoing development and improvement of the data platform
• Ensure data remains securely hosted on MoD-owned infrastructure
• Collaborate with wider technical and delivery teams
• Troubleshoot and resolve technical issues across existing data pipelines

Required Experience

• Strong commercial experience as a Data Engineer
• Experience working with Azure
• Experience with Kubernetes
• Hands-on experience with Apache Airflow and Apache Spark
• Experience building and maintaining data pipelines
• Experience working within government, defence or other secure environments
• Strong understanding of data engineering principles and best practices