DV Data Engineering Lead - London

LA International Computer Consultants Ltd; Greater London; On-site; GBP 80,000 - 110,000; Full time

Job summary

LA International Computer Consultants Ltd seeks a Data Engineering Lead in London for a 4 days onsite role. You’ll design, build and deliver data products, develop data pipelines, and work with open‑source tech to meet operational needs.

The role emphasizes data cataloguing, CI/CD, containerisation with Kubernetes, and Azure Stack Hub. Security clearance candidates are welcome and may require formal clearance before appointment.

Qualifications

• Proficiency in Python and open‑source data engineering technologies.
• Experience building and maintaining data pipelines and data products.
• Ability to design scalable data solutions using Linux/Unix environments.
• Knowledge of CI/CD practices and containerisation (Kubernetes).
• Exposure to cloud/data tooling such as Azure Stack Hub, OpenSearch, Kafka.

Responsibilities

• Design, build and deliver new data products to meet operational needs.
• Develop, maintain and improve data pipelines.
• Utilise open-source technologies to craft practical data solutions.
• Build solutions from ground up within tight delivery timelines.
• Contribute to data cataloguing and asset organisation.
• Adapt designs as operational priorities shift.

Skills

Python

Open-source data engineering

Data pipelines

Linux/Unix

CI/CD

Containerisation

Kubernetes

Apache Airflow

OpenSearch

Azure

Azure Stack Hub

Apache Kafka

Apache NiFi

Graph databases

Tools

Apache Airflow

OpenSearch

Azure

Azure Stack Hub

Apache Kafka

Data cataloguing tools

Apache NiFi

Graph databases

Job description

Data Engineering Lead

4 days onsite in London

6 Months initial contract

Key Responsibilities:

Data engineering and product delivery

• -Design, build and deliver new data products in response to operational needs
• -Develop, maintain and improve data pipelines.
• -Use predominantly open-source technologies to create practical data solutions.
• -Build solutions from the ground up within short and potentially changing delivery timescales.
• -Contribute to data cataloguing and the effective organisation of data assets.
• -Adapt designs and engineering priorities when operational requirements change.

Platforms and engineering practices

• -Contribute to the platform and infrastructure supporting data products.
• -Work in Linux or Unix environments.
• -Apply continuous integration and continuous delivery practices.
• -Use containerisation and orchestration technologies when developing and deploying solutions.
• -Support engineering within an Azure Stack Hub environment.
• -Evaluate or work with emerging technologies where relevant, including pipeline orchestration and graph database tools.

Essential

• -Python
• -Open-source data engineering technologies
• -Data pipeline development
• -Linux or Unix
• -Continuous integration and continuous delivery
• -Containerisation
• -Kubernetes

Preferred

• -Apache Airflow
• -OpenSearch
• -Azure
• -Azure Stack Hub
• -Apache Kafka
• -Data cataloguing tools and practices
• -Apache NiFi
• -Graph databases

Due to the nature and urgency of this post, candidates holding or who have held high level security clearance in the past are most welcome to apply. Please note successful applicants will be required to be security cleared prior to appointment which can take a minimum 18 weeks. LA International is an award-winning partner of choice for many of the world's most influential companies and government organisations. Holding Enhanced Government Security Accreditation, we are recognised as the European market leader in the delivery of Security Cleared talent to organisations that demand the very highest levels of security, compliance and assurance.

An award-winning organisation, having secured the prestigious Queens Award for Enterprise: International Trade over multiple years. We are committed to fostering an inclusive, equitable and accessible workplace where everyone feels valued and supported. We welcome applications from all individuals, regardless of background or identity, and we encourage candidates who may not meet every listed requirement to still apply. If you require any adjustments or support during the recruitment process, please let us know and we will work with you to ensure a fair and accessible experience.

Please Note: If a high volume of applications is received, only candidates shortlisted will be contacted.