Senior Data Engineer

Spectrum.Life; Manchester; Hybrid; GBP 70,000 - 90,000; Full time

Job summary

A health technology firm in the UK is seeking a Senior Data Engineer to lead the development of an AI-driven application. The successful candidate will have extensive experience in data engineering, machine learning, and serverless backend development. This role offers autonomy to shape architecture and deliver impactful solutions. Competitive salary, 25 days annual leave, and the opportunity to work remotely as part of a dynamic team are included.

Benefits

Competitive salary and benefits package

25 days of annual leave

Remote work opportunity

24/7 EAP and wellness supports

Qualifications

• Minimum 5 years of experience in data engineering and 3 years on ML/AI projects.
• Proven track record of delivering complex end-to-end web applications.
• Proficiency in Typescript for building scalable serverless applications.
• Experience designing ETL/ELT pipelines in AWS.
• Familiarity with LLM frameworks like LangChain is desirable.

Responsibilities

• Architect, build, and deploy a solution encompassing data pipelines and serverless backend services.
• Develop clean, secure, and performant backend services.
• Design and implement resilient ETL pipelines within the AWS ecosystem.
• Engineer core application logic by integrating LLMs.

Skills

Data Engineering

Machine Learning

Full-Stack Proficiency

Backend Engineering

Serverless Applications

Generative AI

Communication

Tools

AWS Lambda

AWS API Gateway

S3

Apache Hive

Hadoop

Redshift

Spark

Kafka

Scala

Job description

About Us

is a whole-of-health digital partner that guides organisations and their people to thrive delivering clinically backed digital health mental health and wellbeing solutions.

Our HealthTech delivers digital transformation for Insurers Educators and Employers through Co-creation or seamlessly integrated out-of-the-box solutions that decrease digital fragmentation and engage empower and transform their peoples lives.

Established in 2018 by Stuart McGoldrick and Stephen Costello provides services internationally to over 7.2m insurance members 3000 corporate clients 60 universities and 650000 university students. currently employs over 350 people.

Our vision is to change and save as many lives as possible.

Role Brief

We are seeking an exceptional Senior Data Engineer to lead the development of a sophisticated Generative AI application from the ground up. This project involves creating a robust ETL pipeline and building a scalable serverless backend. You will integrate and orchestrate LLMs to power the core functionality. You will have the autonomy to shape the architecture and deliver a production-ready solution that will have a significant impact.

You are a pragmatic self-managing engineer who thrives on ownership and is passionate about building high-quality software. You are comfortable working autonomously in a remote / hybrid setup and are driven by the challenge of solving complex problems and delivering tangible results.

• Cutting-Edge Technology : Get hands-on experience in one of the most exciting fields in tech today. This project is a fantastic opportunity to deepen your expertise in applied Generative AI.
• Technical Excellence : Uphold high standards for code quality testing and documentation. You will be responsible for the technical integrity of the project.

Responsibilities

• End-to-End Ownership : Architect build and deploy a complete solution encompassing data pipelines serverless backend services and a user-facing React application.
• Serverless Backend Development : Develop clean secure and performant backend services by deploying serverless functions.
• ETL & Data Architecture : Design and implement resilient and scalable ETL pipelines within our AWS ecosystem to ingest and process datasets.
• Generative AI Integration & Orchestration : Engineer the core application logic by integrating and managing interactions with LLMs.

Requirements

• Minimum 5 years of experience on data engineering and 3 years on ML / AI projects.
• Senior-Level Full-Stack Proficiency : A proven track record of delivering complex end-to-end web applications.
• Backend Engineering : Proficiency in and Typescript for building scalable serverless applications. Strong experience in creating and consuming RESTful APIs.
• Serverless Infrastructure : Hands-on expertise designing and building solutions with AWS Lambda AWS API Gateway and S3. Demonstrable experience designing and building ETL / ELT pipelines in AWS.
• Generative AI & LLM Engineering : Experience building applications that call and interact with LLMs. Practical knowledge of using AWS Bedrock or AWS SageMaker to integrate AI models.
• Communication : Excellent written and verbal communication skills. You can articulate complex technical concepts clearly and effectively.

Desirable

• LLM Frameworks : Practical experience with LLM orchestration frameworks like LangChain.
• Data Warehousing : Knowledge of data warehousing concepts and technologies.
• Infrastructure as Code (IaC) : Experience with tools like AWS CDK or Terraform.

Benefits

• Full-time contract
• Competitive salary and benefits package.
• 25 days of annual leave to exclude the public holidays
• Opportunity to work remotely as part of a dynamic and supportive healthcare team.
• 24 / 7 EAP and a wide range of health and wellbeing supports

Extensive list of employee perks and benefits Experience :

Senior IC

Key Skills Apache Hive,S3,Hadoop,Redshift,Spark,AWS,Apache Pig,NoSQL,Big Data,Data Warehouse,Kafka,Scala

Employment Type : Full-Time

Experience : years

Vacancy : 1