Data Engineer: Real-Time Pipelines & Data Security

Sanderson Government & Defence; Woking; On-site; GBP 55,000 - 75,000; Full time

Job summary

Sanderson Government & Defence is seeking a Data Engineer to design and operate data pipelines, transforming raw data into actionable insights for national security customers. You will handle diverse sources, batch, streaming, and unstructured data, applying distributed compute techniques to large-scale workloads.

You will work with an operational team to diagnose data issues, ensure data quality and security, and help decompose user needs into epics and stories for the scrum.

Responsibilities

• Data pipeline development - Data ingestion and pipeline orchestration design and tooling.
• Database schema design / database modelling
• Data integration - Integrating and enriching data from various sources, ensuring data consistency and quality.
• ETL processing design and coding - Extract transform and load processing such as NiFI.
• Create code that is open by default and easy for others to reuse
• Maintain and develop existing architectural components including Data Ingest and Data Stores
• Work as part of an operational team investigating and diagnosing problem identified with integrated (enriched) data.
• Explain the difference between user needs and the desires of the user
• Data security - Implementing data security measures to protect sensitive information.
• Be able to help the scrum team decompose user requests and key results into epics and stories.
• Writing clean, secure code following a test-driven approach
• Monitor and maintain - Monitor data systems for performance issues and make any necessary updates.

Job description

Sanderson Government & Defence is seeking a Data Engineer to design and operate data pipelines, transforming raw data into actionable insights for national security customers. You will handle diverse sources, batch, streaming, and unstructured data, applying distributed compute techniques to large-scale workloads.

You will work with an operational team to diagnose data issues, ensure data quality and security, and help decompose user needs into epics and stories for the scrum.