Lead Data Engineer: Scale Data Pipelines & Impact

Kainos; United Kingdom; Sur place; GBP 90 000 - 120 000; Plein temps

Résumé du poste

Kainos is seeking a Lead Data Engineer (Consultant) to design and build data processing and persistence components for large-scale solutions. You will lead hands-on development in agile teams, guiding code quality and alignment with user needs.

You will contribute to non-functional data pipeline requirements, establish best practices, and mentor engineers while collaborating with security and operations teams to ensure readiness and compliance across projects.

Qualifications

• Experience leading a team of engineers in the implementation of data-intensive system components.
• Experience applying standards for design (patterns), development (style guides) and operational readiness (automation, deployment).
• Proficient software development experience in Java, Scala, or Python.
• Strong commercial experience of working with Palantir.
• Software development experience with data-processing platforms from AWS, Azure, GCP, Databricks.
• Expert in SQL or SQL extensions for analytical use case.
• Expert understanding of distributed data stores and data processing frameworks.
• Ability to clearly communicate technical design both in writing and verbally.
• Proficient in designing analytical and operational data models.
• Desirable: Comfortable with Data Warehouse methods and techniques.
• Desirable: AWS/Azure/GCP Certification in Data Services.
• Desirable: Actively shares thoughts on data practices and participates in communities.
• Desirable: Data-driven approach and continuous improvement mindset.

Responsabilités

• Taking responsibility for the development of whole components or subsystems within a team.
• Focusing on hands-on design and development, using open source and commercial platforms.
• Defining and enforcing development best practice and coaching team members to ensure consistency.
• Working with project architects, taking responsibility for non-functional needs of ETL/ELT data processing pipelines such as robustness and performance.
• Taking responsibility for standards and execution of unit and integration testing within the team.
• Taking responsibility for software product due diligence and integration.
• Leading troubleshooting and tuning of activities.
• Working with Operations teams to ensure the application software is operationally ready.
• Working with Security Architects and accreditors to ensure compliance with relevant legal and security requirements.
• Advising customers and managers on estimated effort and technical implications of user stories and journeys.
• Contributing to technical proposals as part of the sales process.
• Managing, coaching and developing a small number of staff, with focus on performance and career development.

Connaissances

Java

Scala

Python

Outils

Palantir

Databricks

AWS

Azure

GCP

Description du poste

Kainos is seeking a Lead Data Engineer (Consultant) to design and build data processing and persistence components for large-scale solutions. You will lead hands-on development in agile teams, guiding code quality and alignment with user needs.

You will contribute to non-functional data pipeline requirements, establish best practices, and mentor engineers while collaborating with security and operations teams to ensure readiness and compliance across projects.