Lead Data Engineer

Kainos; Greater London; On-site; GBP 70,000 - 90,000; Full time

Job summary

Kainos is seeking a Lead Data Engineer (Consultant) to oversee the development of data-intensive components and lead a team. Responsibilities include collaborating with architects for data processing, defining best practices, and ensuring operational readiness.

The ideal candidate will have experience in Java, Scala, or Python, SQL expertise, and a strong interest in AI technologies. This role is crucial for delivering cutting-edge solutions.

Qualifications

• Experience leading a team of engineers in data system components.
• Proficient in Java, Scala, or Python for development.
• Expertise in SQL or SQL extensions for analytics.

Responsibilities

• Develop components or subsystems including design, code, testing.
• Work with project architects for ETL/ELT data processing.
• Manage and coach team staff and ensure operational readiness.

Skills

Leadership

Java

Scala

Python

SQL

Data Processing

Communication

AI Technologies

Tools

Informatica

Azure Databricks

ETL Tools

Job description

Join Kainos

At Kainos, we’re problem solvers, innovators, and collaborators – driven by a shared mission to create real impact. We transform digital services for millions, deliver cutting‑edge Workday solutions, and push the boundaries of technology together.

Lead Data Engineer (Consultant) – Responsibilities

• Take responsibility for the development of whole components or subsystems within a team, including design, code, testing and defect resolution.
• Focus on hands‑on design and development using open‑source and commercial platforms.
• Define and enforce development best practice, coaching team members to ensure consistency.
• Work with project architects on the non‑functional needs of ETL/ELT data‑processing pipelines such as robustness and performance.
• Take responsibility for standards and execution of unit and integration testing performed within the team.
• Manage software product due diligence and integration.
• Lead troubleshooting and tuning of activities.
• Collaborate with Operations teams to ensure the application software is operationally ready.
• Collaborate with Security Architects and accreditors to ensure compliance with relevant legal and security requirements.
• Advises customers, managers and team members on the estimated effort and technical implications of user stories and user journeys.
• Contribute to technical proposals as part of the sales process.
• Manage, coach and develop a small number of staff, focusing on performance and career development while guiding the team to solve problems together.

Minimum (essential) Requirements

• Experience leading a team of engineers in the implementation of data‑intensive system components.
• Experience applying standards for design (patterns), development (style guides) and operational readiness (automation, deployment).
• Proficient software development experience in Java, Scala or Python.
• Experience with data‑processing platforms from vendors such as Informatica, Azure Databricks or equivalent ETL tools.
• Expertise in SQL or SQL extensions for analytical use cases.
• Expert understanding of distributed data stores and data processing frameworks.
• Ability to communicate technical design clearly, both written and verbally.
• Proficiency in designing analytical and operational data models.
• Interest in AI technologies.

Desirable Skills

• Comfortable with Data Warehouse methods and techniques.
• Actively shares thoughts and views on data practices.
• AWS, Azure or GCP certification in Data Services.
• Expertise in continuous improvement and data best practice.
• Participation in development and/or technology communities.
• Practical experience with AI technologies, tools, processes and delivery.