Data Engineer | LA International | London (Hybrid / WFH)

LA International; Greater London; Hybrid; GBP 55,000 - 110,000; Full time

Job summary

LA International is seeking a skilled Data Engineer for a 6-month contract in London, offering hybrid work with the office twice per week. The role focuses on real-time data ingestion, streaming architectures, and backend processing using Kafka, Kubernetes, Python and Java.

SC clearance preferred and rates are Inside IR35 negotiable. The ideal candidate will design, implement and optimize high-performance data pipelines, ensure data integrity, and collaborate with analytics, data science and

Qualifications

• Bachelor's degree in CS/DE/SE or related field.
• 4+ years in data engineering, backend development, or streaming architectures.
• Hands-on Kafka, distributed messaging expertise.
• Kubernetes and cloud deployment pipelines.
• Python and Java for data processing.

Responsibilities

• Design and build robust data ingestion and streaming pipelines with Kafka.
• Deploy, manage, and scale containerized data processing in Kubernetes.
• Write clean Python/Java for ETL and backend services.
• Architect event-driven real-time analytics and data distribution.
• Troubleshoot pipeline failures and latency bottlenecks in Kafka.
• Collaborate with cross-functional teams and stakeholders.
• Maintain technical docs for data models and runbooks.
• Adhere to security, privacy, and governance standards.

Skills

Apache Kafka

Kubernetes

Python

Java

Data engineering

Backend development

English communication

Hybrid work

Education

Bachelor’s degree in Computer Science or related field

Tools

AWS

Azure

GCP

Terraform

CI/CD (GitHub Actions/GitLab CI)

Job description

Position Summary

LA International is actively seeking an experienced, high-caliber, and technical Data Engineer with solid professional experience in distributed data pipelines, streaming architectures, and backend software development to join our client’s technology team on an initial 6-month contract basis with long-term extension potential in London, UK (Hybrid model: London office twice per week, remaining days remote/WFH; SC clearance preferred). In this critical data engineering role, you will be responsible for designing, building, and optimizing high-performance data pipelines, real-time event streaming systems, and scalable backend applications. You will leverage your deep specialist expertise in Apache Kafka, Kubernetes orchestration, Python, and Java to deliver uncompromised data integrity, low-latency processing, and operational excellence. This opportunity offers competitive Inside IR35 negotiable contracting rates, executive technical visibility, and exceptional career growth prospects within a premier global IT recruitment and digital consulting enterprise.

Detailed Job Description

As a Data Engineer supporting LA International in London, UK, you will take full operational, tactical, and technical ownership of real-time data ingestion pipelines, streaming architectures, containerized deployments, and backend data processing systems. Your day-to-day responsibilities include designing and implementing scalable data pipelines utilizing Apache Kafka for event-driven streaming, deploying and managing containerized workloads across Kubernetes clusters, writing clean, high-performance code in Python and Java for data transformation and backend services, collaborating with cross-functional analytics, data science, and software engineering squads, troubleshooting complex data synchronization bottlenecks, ensuring adherence to enterprise data governance standards, and operating under a flexible hybrid work model (London office attendance twice per week). Given the nature of the engagement, active or eligible SC (Security Check) clearance is strongly preferred. You will utilize your proactive troubleshooting and analytical problem-solving skills to resolve complex data anomalies under strict project delivery schedules.

Key Responsibilities

• Design, build, test, and maintain robust, scalable data ingestion and streaming pipelines utilizing Apache Kafka.
• Deploy, manage, and scale containerized data processing applications and microservices across Kubernetes clusters.
• Write clean, optimized, and maintainable code in Python and Java for data ETL workflows and backend system integration.
• Architect resilient event-driven architectures to support real-time analytics, reporting, and enterprise data distribution.
• Troubleshoot, debug, and resolve complex pipeline failures, latency bottlenecks, and consumer lag issues in Kafka.
• Collaborate seamlessly with cross-functional software engineers, data scientists, and business stakeholders.
• Maintain comprehensive technical documentation of data models, pipeline topologies, and deployment runbooks.
• Adhere strictly to enterprise security policies, data privacy protocols, and secure coding practices.

Required Qualifications & Skills

• Bachelor’s degree in Computer Science, Data Engineering, Software Engineering, or a related technical discipline.
• Minimum of 4+ years of progressive professional experience in data engineering, backend development, or streaming architectures.
• Strong, demonstrable hands-on professional expertise in Apache Kafka event streaming and distributed messaging.
• Solid practical proficiency in Kubernetes container orchestration and cloud-native deployment pipelines.
• Comprehensive working knowledge of Python and Java programming languages for data processing and backend development.
• Exceptional analytical, troubleshooting, debugging, and problem-solving capabilities under complex enterprise conditions.
• Excellent written and verbal English communication skills with strong stakeholder collaboration abilities.
• Mandatory legal work eligibility in the UK and willingness to work on a hybrid basis in London (twice weekly).

Nice-to-Have Skills

• Active UK Security Check (SC) clearance or eligibility to undergo SC clearance vetting.
• Hands-on familiarity with cloud data platforms (AWS, Azure, GCP) and data warehousing technologies (Snowflake, BigQuery).
• Experience with infrastructure-as-code tools (Terraform) and CI/CD automation pipelines (GitHub Actions, GitLab CI).
• Prior professional working experience in defense, government, banking, or highly regulated enterprise sectors.

Recruitment Pro Tip

When applying for this Data Engineer position through LA International, ensure your CV explicitly highlights your Apache Kafka and Kubernetes portfolio, details your Python and Java coding projects, and clarifies your SC clearance status and hybrid London availability.