Senior Data Engineer

Advanced Resource Managers Ltd; Bristol; On-site; GBP 60,000 - 80,000; Full time

Job summary

A large Aerospace firm in Bristol seeks a Senior Data Engineer to design and maintain data pipelines. The ideal candidate will have experience in government or regulated sectors, and proficiency with the Elastic Stack, Apache NiFi, and ETL tools. Key responsibilities include managing data ingestion, collaborating for compliance, and creating dashboards in Kibana. This 12-month contract offers up to £79 per hour, outside of IR35 regulations.

Qualifications

• Strong background in highly regulated industries is a must.
• Proficiency in data processing and ETL tools required.
• Familiarity with containerization and orchestration tools beneficial.

Responsibilities

• Design and maintain secure data pipelines using Elastic Stack.
• Implement data ingestion and transformation processes.
• Collaborate with teams for data governance and security compliance.
• Manage real-time data flows ensuring performance and data integrity.
• Build dashboards and visualizations in Kibana.
• Document data architecture and workflows for operational handover.

Skills

Experience in government or highly regulated industries

Data processing and ETL tools like Apache Kafka, Spark, or Hadoop

Containerization and orchestration tools like Docker and Kubernetes

Monitoring and alerting tools such as Prometheus, Grafana, or ELK

Understanding of ML algorithms

Confidence in developing end-to-end solutions

Infrastructure as code (Terraform, Ansible)

Job description

Senior Data Engineer

Bristol

12-Month Contract

Paying up to £79p/h (Outside IR35)

Role Overview

Our client a large Aerospace company is looking for a experienced Senior Data Engineer with to assist with building and managing data pipelines using the Elastic Stack (Elasticsearch, Logstash, Kibana) and Apache NiFi

Key Responsibilities

• Design, develop, and maintain secure and scalable data pipelines using the Elastic Stack (Elasticsearch, Logstash, Kibana) and Apache NiFi.
• Implement data ingestion, transformation, and integration processes, ensuring data quality and security.
• Collaborate with data architects and security teams to ensure compliance with security policies and data governance standards.
• Manage and monitor large-scale data flows in real-time, ensuring system performance, reliability, and data integrity.
• Develop robust data models to support analytics and reporting within secure environments.
• Perform troubleshooting, debugging, and performance tuning of data pipelines and the Elastic Stack.
• Build dashboards and visualizations in Kibana to enable data-driven decision-making.
• Ensure high availability and disaster recovery for data systems, implementing appropriate backup and replication strategies.
• Document data architecture, workflows, and security protocols to ensure smooth operational handover and audit readiness.

Required Skillset

• Experience working in government, defence, or highly regulated industries with knowledge of relevant standards.
• Experience with additional data processing and ETL tools like Apache Kafka, Spark, or Hadoop
• Familiarity with containerization and orchestration tools such as Docker and Kubernetes.
• Experience with monitoring and alerting tools such as Prometheus, Grafana, or ELK for data infrastructure.
• Understanding of ML algorithms, their development and implementation
• Confidence in developing end-to-end solutions
• Experience with infrastructure as code e.g. Terraform, Ansible