Senior Data Engineer: Real-Time Data Pipelines(Elastic/NiFi)

Advanced Resource Managers Ltd; Bristol; On-site; GBP 60,000 - 80,000; Full time

Job summary

A large Aerospace firm in Bristol seeks a Senior Data Engineer to design and maintain data pipelines. The ideal candidate will have experience in government or regulated sectors, and proficiency with the Elastic Stack, Apache NiFi, and ETL tools. Key responsibilities include managing data ingestion, collaborating for compliance, and creating dashboards in Kibana. This 12-month contract offers up to £79 per hour, outside of IR35 regulations.

Qualifications

• Strong background in highly regulated industries is a must.
• Proficiency in data processing and ETL tools required.
• Familiarity with containerization and orchestration tools beneficial.

Responsibilities

• Design and maintain secure data pipelines using Elastic Stack.
• Implement data ingestion and transformation processes.
• Collaborate with teams for data governance and security compliance.
• Manage real-time data flows ensuring performance and data integrity.
• Build dashboards and visualizations in Kibana.
• Document data architecture and workflows for operational handover.

Skills

Experience in government or highly regulated industries

Data processing and ETL tools like Apache Kafka, Spark, or Hadoop

Containerization and orchestration tools like Docker and Kubernetes

Monitoring and alerting tools such as Prometheus, Grafana, or ELK

Understanding of ML algorithms

Confidence in developing end-to-end solutions

Infrastructure as code (Terraform, Ansible)

Job description

A large Aerospace firm in Bristol seeks a Senior Data Engineer to design and maintain data pipelines. The ideal candidate will have experience in government or regulated sectors, and proficiency with the Elastic Stack, Apache NiFi, and ETL tools. Key responsibilities include managing data ingestion, collaborating for compliance, and creating dashboards in Kibana. This 12-month contract offers up to £79 per hour, outside of IR35 regulations.