Data Engineer (HS)

Expleo; City Of London; On-site; GBP 50,000 - 70,000; Full time

Job summary

A technology consulting firm based in London is seeking a Data Engineer to support health assessment outcomes and AI personalisation projects. The role involves designing and maintaining robust data pipelines, optimizing extraction processes, and ensuring data integrity. Candidates should have expert SQL and Python skills, with proficiency in handling diverse data sources and working with tools like GCP and Azure. The position offers a collaborative environment and opportunities for professional development.

Benefits

Collaborative working environment

Access to ExpleoAcademy training courses

Competitive company benefits

Qualifications

• Proficient in ETL/ELT methodologies for efficient data processing.
• Experience creating data pipelines from scratch along the entire data chain.

Responsibilities

• Review and optimise current data extraction processes.
• Refactor SQL scripts into modular, reusable units.
• Evaluate current data pipeline architecture for efficiency.
• Implement structured orchestration for data workflows.
• Introduce versioning practices to ensure reproducibility.

Skills

Expert-level SQL proficiency

Python fluency

Proficient in handling data from diverse sources

Solid understanding of modern EDW cloud tools

Proficiency in GCP services

Tools

Azure technologies

Snowflake

Job description

Overview

We are seeking a Data Engineer to support the implementation of Health Assessment Outcomes and AI Personalisation Projects. These initiatives are part of a multi-year programme aimed at delivering measurable impact through advanced clinical statistical techniques and innovative AI solutions. The successful candidate will design, build, and maintain robust data pipelines and infrastructure that underpin our data platforms and analytics capabilities

Responsibilities

• Review and optimise current data extraction processes, identifying opportunities for performance improvements and reliability.
• Refactor SQL scripts into modular, reusable units to improve maintainability and scalability across projects.
• Evaluate current data pipeline architecture to ensure robustness, efficiency, and alignment with clinical data requirements.
• Implement structured orchestration for previously manual or ad hoc data workflows.
• Introduce versioning practices to track changes and ensure reproducibility.
• Establish testing frameworks to validate data integrity and pipeline performance.
• Define and enforce consistent output schemas to facilitate downstream integration and reuse.
• Develop shared code libraries and maintain comprehensive metadata and documentation to support collaboration and transparency.
• Design and deploy observability tools and practices to monitor, log, and troubleshoot clinical data pipelines effectively.

Qualifications

• Expert-level SQL proficiency and Python fluency.

Essential skills

• Proficient in handling data from diverse sources such as flat files, APIs, and databases using ETL/ELT methodologies for efficient data processing.
• Solid understanding of modern EDW cloud tools, techniques, and best practices.
• Proficiency in GCP services such as BigQuery, Cloud Storage, Dataflow, and Pub/Sub for building scalable, cloud-native data pipelines.

Experience

• Experience in Azure technologies and Snowflake.
• Experience creating data pipelines from scratch along the entire data chain, including implementing data load patterns, error-handling, and automated procedures.

Benefits

• Collaborative working environment – we stand shoulder to shoulder with our clients and ourpeers through good times and challenges
• We empower all passionate technology loving professionals by allowing them to expand their skills and take part in inspiring projects
• ExpleoAcademy - enables you to acquire and develop the right skills by delivering a suite of accredited training courses
• Competitive company benefits
• Always working as one team, our people are not afraid to think big and challenge the status quo

• As a Disability Confident Committed Employer we have committed to:
• Ensure our recruitment process is inclusive and accessible
• Communicating and promoting vacancies
• Offering an interview to disabled people who meet the minimum criteria for the job
• Anticipating and providing reasonable adjustments as required
• Supporting any existing employee who acquires a disability or long term health condition, enabling them to stay in work at least one activity that will make a difference for disabled people

“We are an equal opportunities employer and welcome applications from all suitably qualified persons regardless of their race, sex, disability, religion/belief, sexual orientation or age”.

We treat everyone fairly and equitably across the organisation, including providing any additional support and adjustments needed for everyone to thrive

#LI-BM1

#LI-DS1