DV Data Engineering Lead - London

LA International; City Of London; On-site; GBP 90,000 - 140,000; Full time

Job summary

LA International seeks a Data Engineering Lead for a London-based, 6-month contract, onsite four days a week. You will lead data product delivery and design scalable pipelines.

You will work with Linux/Unix environments, CI/CD, containers and Kubernetes, and leverage Open-Source data tooling and cloud integration.

Candidates with security clearance or eligibility will be favoured; the role requires flexibility to meet tight delivery timelines.

Qualifications

• Experience designing and delivering data products for operational needs.
• Strong data pipelines development using open-source tools.
• Proficiency with Linux/Unix environments and container orchestration.
• Experience with CI/CD pipelines.
• Ability to work on short or changing delivery timelines.
• Security clearance friendly; ability to obtain clearance.

Responsibilities

• Design, build and deliver new data products in response to operational needs.
• Develop, maintain and improve data pipelines.
• Use predominantly open-source technologies to create practical data solutions.
• Build solutions from the ground up within short and potentially changing delivery timescales.
• Contribute to data cataloguing and the effective organisation of data assets.
• Adapt designs and engineering priorities when operational requirements change.

Skills

Python

Open-source data eng technologies

Data pipeline development

Linux/Unix

CI/CD

Containerisation

Kubernetes

Tools

Apache Airflow

OpenSearch

Azure

Azure Stack Hub

Apache Kafka

Data cataloguing tools

Apache NiFi

Graph databases

Job description

Data Engineering Lead

4 days onsite in London

6 Months initial contract

Key Responsibilities:

Data engineering and product delivery

• -Design, build and deliver new data products in response to operational needs
• -Develop, maintain and improve data pipelines.
• -Use predominantly open-source technologies to create practical data solutions.
• -Build solutions from the ground up within short and potentially changing delivery timescales.
• -Contribute to data cataloguing and the effective organisation of data assets.
• -Adapt designs and engineering priorities when operational requirements change.

Platforms and engineering practices

• -Contribute to the platform and infrastructure supporting data products.
• -Work in Linux or Unix environments.
• -Apply continuous integration and continuous delivery practices.
• -Use containerisation and orchestration technologies when developing and deploying solutions.
• -Support engineering within an Azure Stack Hub environment.
• -Evaluate or work with emerging technologies where relevant, including pipeline orchestration and graph database tools.

Essential

• -Python
• -Open-source data engineering technologies
• -Data pipeline development
• -Linux or Unix
• -Continuous integration and continuous delivery
• -Containerisation
• -Kubernetes

Preferred

• -Apache Airflow
• -OpenSearch
• -Azure
• -Azure Stack Hub
• -Apache Kafka
• -Data cataloguing tools and practices
• -Apache NiFi
• -Graph databases

Due to the nature and urgency of this post, candidates holding or who have held high level security clearance in the past are most welcome to apply. Please note successful applicants will be required to be security cleared prior to appointment which can take a minimum 18 weeks. LA International is an award-winning partner of choice for many of the world's most influential companies and government organisations. Holding Enhanced Government Security Accreditation, we are recognised as the European market leader in the delivery of Security Cleared talent to organisations that demand the very highest levels of security, compliance and assurance.

An award-winning organisation, having secured the prestigious Queens Award for Enterprise: International Trade over multiple years. We are committed to fostering an inclusive, equitable and accessible workplace where everyone feels valued and supported. We welcome applications from all individuals, regardless of background or identity, and we encourage candidates who may not meet every listed requirement to still apply. If you require any adjustments or support during the recruitment process, please let us know and we will work with you to ensure a fair and accessible experience.

Please Note: If a high volume of applications is received, only candidates shortlisted will be contacted.