Senior Data Engineer

Arm; United Kingdom; On-site; GBP 60,000 - 80,000; Full time

Job summary

A leading aerospace consultancy in the UK is seeking a Senior Data Engineer for a 12-month contract in Bristol, paying up to £79 per hour. The role involves building and managing secure data pipelines using the Elastic Stack and Apache NiFi. Candidates should have experience in regulated industries, knowledge of data processing tools, and familiarity with containerization. This position supports rapid and reliable data-driven decision-making in a fast-paced environment.

Qualifications

• Experience working in regulated industries.
• Knowledge of data processing and ETL tools.
• Familiarity with containerization tools.

Responsibilities

• Develop and maintain secure data pipelines.
• Implement data ingestion and transformation processes.
• Collaborate with architects and security teams.

Skills

Data processing and ETL tools

Technical compliance awareness

Containerization and orchestration

Monitoring and alerting tools

Understanding of machine learning algorithms

End-to-end solution development

Infrastructure as code expertise

Tools

Elastic Stack

Apache NiFi

Docker

Kubernetes

Apache Kafka

Spark

Hadoop

Prometheus

Grafana

Job description

Senior Data Engineer

Bristol

12-Month Contract

Paying up to 79p/h (Outside IR35)

Role Overview

Our client a large Aerospace company is looking for a experienced Senior Data Engineer to assist with building and managing data pipelines using the Elastic Stack (Elasticsearch, Logstash, Kibana) and Apache NiFi

Key Responsibilities

• Design, develop, and maintain secure and scalable data pipelines using the Elastic Stack (Elasticsearch, Logstash, Kibana) and Apache NiFi.
• Implement data ingestion, transformation, and integration processes, ensuring data quality and security.
• Collaborate with data architects and security teams to ensure compliance with security policies and data governance standards.
• Manage and monitor large-scale data flows in real-time, ensuring system performance, reliability, and data integrity.
• Develop robust data models to support analytics and reporting within secure environments.
• Perform troubleshooting, debugging, and performance tuning of data pipelines and the Elastic Stack.
• Build dashboards and visualizations in Kibana to enable data-driven decision-making.
• Ensure high availability and disaster recovery for data systems, implementing appropriate backup and replication strategies.
• Document data architecture, workflows, and security protocols to ensure smooth operational handover and audit readiness.

Required Skillset

• Experience working in government, defence, or highly regulated industries with knowledge of relevant standards.
• Experience with additional data processing and ETL tools like Apache Kafka, Spark, or Hadoop
• Familiarity with containerization and orchestration tools such as Docker and Kubernetes.
• Experience with monitoring and alerting tools such as Prometheus, Grafana, or ELK for data infrastructure.
• Understanding of ML algorithms, their development and implementation
• Confidence in developing end-to-end solutions
• Experience with infrastructure as code e.g. Terraform, Ansible

This vacancy is being advertised by either Advanced Resource Managers Limited, Advanced Resource Managers IT Limited or Advanced Resource Managers Engineering Limited ("ARM"). ARM is a specialist talent acquisition and management consultancy. We provide technical contingency recruitment and a portfolio of more complex resource solutions. Our specialist recruitment divisions cover the entire technical arena, including some of the most economically and strategically important industries in the UK and the world today. We will never send your CV without your permission. Where the role is marked as Outside IR35 in the advertisement this is subject to receipt of a final Status Determination Statement from the end Client and may be subject to change.