Data Eng Lead — Open-Source Pipelines, London

LA International; City Of London; On-site; GBP 90,000 - 140,000; Full time

Job summary

LA International seeks a Data Engineering Lead for a London-based, 6-month contract, onsite four days a week. You will lead data product delivery and design scalable pipelines.

You will work with Linux/Unix environments, CI/CD, containers and Kubernetes, and leverage Open-Source data tooling and cloud integration.

Candidates with security clearance or eligibility will be favoured; the role requires flexibility to meet tight delivery timelines.

Qualifications

• Experience designing and delivering data products for operational needs.
• Strong data pipelines development using open-source tools.
• Proficiency with Linux/Unix environments and container orchestration.
• Experience with CI/CD pipelines.
• Ability to work on short or changing delivery timelines.
• Security clearance friendly; ability to obtain clearance.

Responsibilities

• Design, build and deliver new data products in response to operational needs.
• Develop, maintain and improve data pipelines.
• Use predominantly open-source technologies to create practical data solutions.
• Build solutions from the ground up within short and potentially changing delivery timescales.
• Contribute to data cataloguing and the effective organisation of data assets.
• Adapt designs and engineering priorities when operational requirements change.

Skills

Python

Open-source data eng technologies

Data pipeline development

Linux/Unix

CI/CD

Containerisation

Kubernetes

Tools

Apache Airflow

OpenSearch

Azure

Azure Stack Hub

Apache Kafka

Data cataloguing tools

Apache NiFi

Graph databases

Job description

LA International seeks a Data Engineering Lead for a London-based, 6-month contract, onsite four days a week. You will lead data product delivery and design scalable pipelines.

You will work with Linux/Unix environments, CI/CD, containers and Kubernetes, and leverage Open-Source data tooling and cloud integration.

Candidates with security clearance or eligibility will be favoured; the role requires flexibility to meet tight delivery timelines.