Senior Data Engineer: Cloud Pipelines for National Security

Copello Global Ltd; Cheltenham; On-site; GBP 80,000 - 100,000; Full time

Job summary

Copello Global Ltd is seeking an experienced Data Engineer to join a growing National Security technology team delivering mission critical data solutions for UK customers.

You will design, develop and maintain data pipelines, work with distributed computing and cloud technologies, and collaborate with customers and engineering teams to transform large datasets into valuable information for analytics and reporting.

Qualifications

• Data engineering and data pipeline development
• Python or Java
• AWS and cloud based data technologies
• Apache NiFi
• Big Data technologies such as Apache Spark, Hadoop or Pig
• Data ingestion, integration and transformation
• Distributed data processing
• Data quality and data security
• Software development and engineering best practices
• Version control and collaborative software development
• Working within Agile or Scrum development environments

Responsibilities

• Design, develop and maintain data pipelines and data ingestion services
• Develop data pipeline orchestration solutions and tooling
• Integrate and enrich data from multiple and diverse sources
• Build reliable and scalable data processing solutions
• Work with large datasets using distributed computing technologies
• Develop data services that support analytics, reporting and downstream applications
• Apply appropriate data security measures when working with sensitive information
• Monitor data systems and identify opportunities for performance optimisation
• Investigate and resolve issues across data pipelines and data systems
• Work as part of an Agile Scrum team
• Help decompose user requirements and key results into epics and user stories
• Write clean, secure and maintainable code
• Follow a test driven development approach
• Develop reusable code and engineering solutions
• Work closely with customers and other technical specialists to understand and solve complex problems

Skills

Data pipelines

Python

AWS

Apache NiFi

Big Data

Distributed computing

Data security

Agile/Scrum

Version control

Data ingestion

Job description

Copello Global Ltd is seeking an experienced Data Engineer to join a growing National Security technology team delivering mission critical data solutions for UK customers.

You will design, develop and maintain data pipelines, work with distributed computing and cloud technologies, and collaborate with customers and engineering teams to transform large datasets into valuable information for analytics and reporting.