Lead Data Engineer: Architect Scalable Data Pipelines

Kainos; Belfast City District; On-site; GBP 90,000 - 130,000; Full time

Job summary

Kainos is seeking a Lead Data Engineer (Consultant) in the United Kingdom to design and develop data processing and persistence components for data-at-scale solutions. You will lead hands-on development, define standards, and mentor a small team in an Agile environment.

You will work with project architects, security teams and operations to ensure robustness, performance, and compliance while delivering high-quality data solutions for enterprise clients.

Qualifications

• Experience leading a team of engineers in data-intensive components.
• Proven ability to apply design patterns, coding standards and automation.
• Proficient in Java, Scala or Python with strong SQL skills.
• Extensive experience with Palantir and cloud data platforms (AWS/Azure/GCP).
• Deep understanding of distributed data stores and data processing frameworks.
• Excellent written and verbal communication; able to convey technical design clearly.

Responsibilities

• Lead development of whole components or subsystems within a team, including design, code, tests and defect resolution.
• Hands-on design and development using open source and commercial platforms.
• Set development standards, enforce best practices, and coach teammates.
• Own non-functional needs of ETL/ELT pipelines for robustness and performance.
• Oversee unit and integration testing and ensure readiness for deployment.
• Contribute to technical proposals during the sales process.
• Manage and coach a small group of staff and guide career development.
• Collaborate with project architects, security teams and operations for operational readiness and compliance.

Skills

Team leadership

Java

Scala

Python

Palantir

Cloud data platforms (AWS/Azure/GCP)

SQL

Distributed data stores

Data modeling

Tools

AWS

Azure

GCP

Databricks

Job description

Kainos is seeking a Lead Data Engineer (Consultant) in the United Kingdom to design and develop data processing and persistence components for data-at-scale solutions. You will lead hands-on development, define standards, and mentor a small team in an Agile environment.

You will work with project architects, security teams and operations to ensure robustness, performance, and compliance while delivering high-quality data solutions for enterprise clients.