Data Engineer (Java, AWS, Kafka, Kubernetes)

Skillfinder International; Bannockburn; Hybrid; GBP 90,000 - 110,000; Full time

Job summary

Skillfinder International is seeking a Senior Data Engineer in London (Hybrid) to work on a high-profile UK Government AI & Data programme. You will apply a strong Java software engineering background to deliver secure, data-driven solutions in a cloud-native environment.

The role combines hands-on software engineering with data engineering, focusing on building scalable data pipelines, ETL processes and data integration using AWS, Kafka and Kubernetes.

Qualifications

• Advanced Java development experience and delivery of production systems.
• Strong commercial experience in Data Engineering.
• Experience designing data pipelines, ETL processes and data integration solutions.
• Strong software engineering fundamentals for scalable, maintainable apps.
• Knowledge of data modelling, data quality and data lifecycle management.
• Experience in Agile development teams.
• Active UK SC security clearance.

Responsibilities

• Design, develop and maintain scalable data pipelines and processing solutions.
• Build reliable, high-performance applications using Java.
• Develop and enhance data ingestion, transformation and integration processes.
• Work with cloud-based infrastructure and services, particularly AWS.
• Support event-driven and distributed processing architectures.
• Apply data modelling, data quality and data lifecycle management principles.
• Collaborate with software engineers, data engineers, architects and other delivery teams.
• Contribute to technical design, code reviews, testing and continuous improvement.
• Deliver solutions using Agile engineering practices.

Skills

Advanced Java

Data Engineering

Data pipelines

ETL

Agile

Data modelling

Data quality

Data lifecycle

Security clearance

Tools

AWS

Apache Kafka

Kubernetes

Job description

Senior Data Engineer – Java, AWS, Kafka, Kubernetes

SC Cleared | London (Hybrid) | Inside IR35 | Contract until December 2027

Are you an experienced Senior Data Engineer with a strong Java software engineering background looking to work on a high-profile UK Government AI & Data programme?

We're recruiting for a Senior Data Engineer to support a major public sector organisation delivering critical data solutions within a secure environment.

This is an opportunity to work with modern cloud-native technologies, build scalable data processing solutions and contribute to systems that support essential public services.

The Role

You'll work within an established engineering environment, developing and maintaining robust data solutions using Java and modern cloud technologies.

The role combines hands-on software engineering with data engineering, making it particularly suitable for someone who has developed production-grade Java applications alongside data pipelines and distributed processing systems.

Key Responsibilities

• Design, develop and maintain scalable data pipelines and processing solutions.

• Build reliable, high-performance applications using Java.

• Develop and enhance data ingestion, transformation and integration processes.

• Work with cloud-based infrastructure and services, particularly AWS.

• Support event-driven and distributed processing architectures.

• Apply data modelling, data quality and data lifecycle management principles.

• Collaborate with software engineers, data engineers, architects and other delivery teams.

• Contribute to technical design, code reviews, testing and continuous improvement.

• Deliver solutions using Agile engineering practices.

Essential Skills and Experience

• Advanced Java development experience, including hands-on delivery of production systems.

• Strong commercial experience in Data Engineering.

• Experience designing and developing data pipelines, ETL processes and data integration solutions.

• Strong software engineering fundamentals and experience building scalable, maintainable applications.

• Knowledge of data modelling, data quality and data lifecycle management.

• Experience working within Agile development teams.

• Active UK SC security clearance.

Highly Desirable

• AWS – experience building or supporting cloud-native data solutions.

• Apache Kafka – experience with event streaming, messaging or real-time data processing.

• Kubernetes – experience with container orchestration and distributed application deployment.

• Experience developing Java-based data processing or event-driven applications.

Additional Advantageous Experience

• Apache Spark, Flink, Hadoop or other Big Data technologies.

• Working with high-volume or complex enterprise datasets.

• Experience delivering solutions within government, defence or other secure environments.

Why Consider This Opportunity?

• Long-term contract running until December 2027.

• Work on a significant UK public sector AI & Data programme.

• Modern technology stack involving Java, AWS, Kafka and Kubernetes.

• Opportunity to contribute to secure, business-critical data platforms.

• Hybrid London working arrangement.

Please note: Active SC clearance is required for this position. The role requires three days onsite in London each week and falls inside IR35.