Lead Data Engineer

Limelight Health; Belfast City District; On-site; GBP 90,000 - 130,000; Full time

Job summary

Kainos in Belfast seeks a Lead Data Engineer (Consultant) to design and deliver scalable data processing and persistence components. You will lead hands-on development in Java, Scala or Python, mentor engineers, and ensure code quality aligns with user needs in an agile setting.

You will own non-functional requirements for ETL/ELT pipelines, drive testing standards, collaborate with project architects, security and operations to ensure operational readiness and regulatory compliance.

Qualifications

• Experience leading a team of engineers in data-intensive components.
• Knowledge of design patterns, coding standards, and deployment readiness.
• Proficient in Java, Scala, or Python.
• Experience with data-processing platforms (Informatica, Azure Databricks or ETL tools).
• Expert in SQL for analytical use cases.
• Understanding of distributed data stores and processing frameworks.

Responsibilities

• Take responsibility for the development of whole components or subsystems within a team (design, code, test, defects).
• Hands-on design and development using open source and commercial platforms.
• Define and enforce development best practices and coach team members.
• Lead non-functional aspects of ETL/ELT pipelines (robustness, performance).
• Oversee unit and integration testing standards within the team.
• Perform software product due diligence and integration tasks.
• Lead troubleshooting and tuning activities.
• Collaborate with Operations to ensure operational readiness of applications.
• Coordinate with Security Architects and accreditors for compliance.
• Advise customers and managers on effort estimates and technical implications.
• Contribute to technical proposals as part of sales process.
• Manage, coach and develop a small staff, providing direction for the team.

Skills

Team leadership

Java/Scala/Python

ETL/ELT

SQL

Distributed data stores

Analytical data modeling

Communication

Tools

Informatica

Azure Databricks

Job description

hackajob is collaborating with Kainos to connect them with exceptional professionals for this role.

Lead Data Engineer

As a Lead Data Engineer (Consultant) at Kainos you will be responsible for designing and developing data processing and data persistence software components for solutions which handle data at scale. Working in agile teams, Lead Data Engineers provide strong development leadership for team members and take responsibility for the quality of the codebase as well as the match to user needs.

Your Responsibilities Will Include

• Taking responsibility for the development of whole components or subsystems within a team. Development incorporates design, code, test and defect resolution.


• Focusing on hands‑on design and development, using open source and commercial platforms.


• Defining and enforcing development best practice and coaching team members to ensure consistency.


• Working with project architects, taking responsibility for non‑functional needs of ETL/ELT data processing pipelines such as robustness and performance.


• Taking responsibility for standards and execution of unit and integration testing done within the team.


• Taking responsibility for software product due diligence and integration.


• Leading troubleshooting and tuning of activities.


• Working with Operations teams to ensure the application software is operationally ready.


• Working with Security Architects and accreditors to ensure compliance with relevant legal and security requirements


• Advising customers and managers and other team members of the estimated effort and technical implications of user stories and user journeys.


• Contributing to technical proposals as part of the sales process.


• Managing, coaching and developing a small number of staff, with a focus on managing employee performance and assisting in their career development. You’ll also provide direction for your team as you solve problems together

Minimum (essential) Requirements

• Experience of leading a team of engineers in the implementation of data‑intensive system components


• Experience of applying standards for design (patterns), development, (style guides) and operational readiness (automation, deployment)


• Proficient software development experience in one of Java, Scala, or Python


• Software development experience with data‑processing platforms from vendors including Informatica, Azure Databricks or any relevant ETL tools


• Expert in SQL or SQL extensions for analytical use case


• Expert understanding of distributed data stores and data processing frameworks


• Ability to simply and clearly communicate technical design both written and verbally


• Proficient in designing analytical and operational data models


• A keen interest in AI technologies

Desirable

• Comfortable with Data Warehouse methods and techniques


• Actively shares their thoughts and views on data practices


• AWS/Azure/GCP Certified in Data Services


• Expertise in continuous improvement and sharing input on data best practice


• Participation in development and/or technology communities


• Practical experience with AI technologies, tools, processes and delivery

Who You Are

Our vision is to enable outstanding people to create digital solutions that have a positive impact on people’s lives. Our values aren't abstract; they are the behaviours we expect from each other every day, and underpin everything that we do. We expect everyone to display our values by being determined in how obstacles are overcome; honest when dealing with others; respectful of how you treat others; creative to find solutions to complex problems and cooperative by sharing information, knowledge and experience.

About Us

Kainos is a high‑growth IT services company providing digital technology solutions and agile software development to enterprise customers. Across our 30‑year history, we have worked on transformational projects across government, NHS and a myriad of private sector clients.