Senior Data Engineer (DV Cleared) - Secure Data Pipelines

ARM; England; Presencial; GBP 52.000 - 85.000; Tempo integral

Resumo da vaga

Advanced Resource Managers Ltd is seeking a Senior Data Engineer to design, build and maintain secure, scalable data solutions for a short-term engagement in Wiltshire.

The role requires active MOD DV clearance, strong Python and SQL skills, and hands-on experience with Apache NiFi, data modelling and OpenShift/Kubernetes to deliver end-to-end data pipelines and resilient data contracts.

Qualificações

• Proven experience delivering data engineering solutions in a project or production environment.
• Strong Python and SQL skills.
• Hands-on experience with Apache NiFi and data pipeline orchestration.
• Experience with data modelling and schema design.
• Experience with Parquet and S3-compatible object storage.
• Experience with REST API integration.
• Experience with OpenShift or Kubernetes.
• Experience with GitLab CI/CD practices and pipelines.
• Experience developing Data contracts or Interface Control Documents (ICDs).
• Experience with data validation and assurance.
• Experience with metadata management, lineage and traceability.

Responsabilidades

• Design, build and maintain secure, scalable data solutions across the full lifecycle, including acquisition, integration, storage, processing and analysis.
• Develop and maintain data ingestion and transformation pipelines using Apache NiFi, Python and SQL.
• Design data models, schemas and data contracts / Interface Control Documents (ICDs) to support reliable integration between systems.
• Integrate data from REST APIs and other source systems, working with Parquet and S3-compatible object storage.
• Implement data validation, assurance, metadata management and traceability to ensure data is accurate, complete and fit for purpose.
• Develop tools for data-driven insights and task automation, and research and adopt appropriate emerging technologies.
• Collaborate with customer and internal stakeholders to define data, interface and tool requirements.
• Work closely with DevOps Engineers, Data Scientists, Analysts and Cyber Consultants to deliver integrated solutions.
• Take ownership of deliverables and ensure the quality, security, and maintainability of outputs.

Conhecimentos

Python

SQL

Data pipeline orchestration

Data modelling

ICDs / data contracts

REST API integration

Parquet/S3 storage

NiFi

OpenShift or Kubernetes

GitLab CI/CD

Ferramentas

Apache NiFi

OpenShift

Kubernetes

GitLab CI/CD

Parquet

S3-compatible storage

REST APIs

Descrição da vaga

Advanced Resource Managers Ltd is seeking a Senior Data Engineer to design, build and maintain secure, scalable data solutions for a short-term engagement in Wiltshire.

The role requires active MOD DV clearance, strong Python and SQL skills, and hands-on experience with Apache NiFi, data modelling and OpenShift/Kubernetes to deliver end-to-end data pipelines and resilient data contracts.