Lead Data Engineer - Scale Data Pipelines & Lead Teams

Limelight Health; Belfast City District; On-site; GBP 90,000 - 130,000; Full time

Job summary

Kainos in Belfast seeks a Lead Data Engineer (Consultant) to design and deliver scalable data processing and persistence components. You will lead hands-on development in Java, Scala or Python, mentor engineers, and ensure code quality aligns with user needs in an agile setting.

You will own non-functional requirements for ETL/ELT pipelines, drive testing standards, collaborate with project architects, security and operations to ensure operational readiness and regulatory compliance.

Qualifications

• Experience leading a team of engineers in data-intensive components.
• Knowledge of design patterns, coding standards, and deployment readiness.
• Proficient in Java, Scala, or Python.
• Experience with data-processing platforms (Informatica, Azure Databricks or ETL tools).
• Expert in SQL for analytical use cases.
• Understanding of distributed data stores and processing frameworks.

Responsibilities

• Take responsibility for the development of whole components or subsystems within a team (design, code, test, defects).
• Hands-on design and development using open source and commercial platforms.
• Define and enforce development best practices and coach team members.
• Lead non-functional aspects of ETL/ELT pipelines (robustness, performance).
• Oversee unit and integration testing standards within the team.
• Perform software product due diligence and integration tasks.
• Lead troubleshooting and tuning activities.
• Collaborate with Operations to ensure operational readiness of applications.
• Coordinate with Security Architects and accreditors for compliance.
• Advise customers and managers on effort estimates and technical implications.
• Contribute to technical proposals as part of sales process.
• Manage, coach and develop a small staff, providing direction for the team.

Skills

Team leadership

Java/Scala/Python

ETL/ELT

SQL

Distributed data stores

Analytical data modeling

Communication

Tools

Informatica

Azure Databricks

Job description

Kainos in Belfast seeks a Lead Data Engineer (Consultant) to design and deliver scalable data processing and persistence components. You will lead hands-on development in Java, Scala or Python, mentor engineers, and ensure code quality aligns with user needs in an agile setting.

You will own non-functional requirements for ETL/ELT pipelines, drive testing standards, collaborate with project architects, security and operations to ensure operational readiness and regulatory compliance.