Lead Data Engineer

DataTruth Ltd; Greater London; Hybrid; GBP 90,000 - 150,000; Full time

Benefits offered by this job

25 days leave

London hybrid

MacBook Pro

Competitive pay

Job summary

DataTruth Ltd in London is seeking a Lead Data Engineer to design, build and operate data platforms and ETL pipelines for client projects. You’ll shape architecture, lead hands‑on delivery, and collaborate with product owners, architects, engineers and QA to deliver robust data solutions.

You'll use cloud tech, AI-accelerated workflows, and modern tooling to solve complex data challenges while mentoring teams and ensuring security and data privacy best practices.

Qualifications

• Deep experience in data architecture and engineering best practices.
• Hands-on lead role with cloud data platforms and ETL pipelines.

Responsibilities

• Collaborate with clients and stakeholders to identify project requirements and define scope.
• Develop secure, robust data and technology solutions that solve real problems.
• Own the design, build, and operation of the data platform, pipelines, infrastructure, and integrations.
• Work with product owners, architects, engineers, designers, QA and other specialists.
• Advise clients on technical options and delivery estimates.
• Lead teams of experts, supporting their development and new challenges.
• Uphold high standards in data engineering, security and privacy.

Skills

SQL

dbt

dlt

data modelling

Python

Git

Docker

ETL pipelines

PostgreSQL

DuckDB

Airflow

Step Functions

Claude Code

Codex

AWS Lambda

AWS ECS

Data testing

Vendor selection

Data warehousing

CDK

CI/CD

Tools

Cloud data platforms

ETL pipelines

AI tooling

Job description

Purposeful Technology

DataTruth is an entrepreneurial technology consultancy where we solve mission-critical problems for ambitious organisations and have fun doing it! If you’re passionate about data engineering, relish the opportunity to turn complex data problems into elegant technical solutions, and want to see your work make a difference, we’d love to hear from you.

Lead Data Engineer

We are looking for an experienced Lead Data Engineer to join us on a permanent or contract basis.

You’ll lead the design, build and operation of data platforms, ETL pipelines, analytics layers and data-driven digital solutions. Working closely with clients and fellow engineers, you’ll shape the architecture and lead hands‑on delivery through to production.

Responsibilities

This role offers a unique blend of strategic and hands‑on work. You'll have the chance to engage with clients and stakeholders, understand their business challenges, and solve their biggest pain points. Day-to-day, you will be:

• Collaborating with clients and stakeholders to identify project requirements and define scope.
• Developing secure, robust data and technology solutions that solve real problems.
• Owning the design, build, and operation of the data platform, pipelines, infrastructure, and integrations.
• Working alongside product owners, architects, engineers, designers, QA and other specialists.
• Advising clients on technical options, providing strategic guidance and contributing to delivery estimates.
• Leading teams of experts, supporting their development and helping them take on new challenges and responsibilities.
• Upholding high standards and best practices in data engineering, security, and data privacy.
  
• Selecting vendors and tooling based on client business requirements, comparative analysis, and justified reasoning.

Your Skills

Your background should include several years of data engineering experience, with the last few spent in a hands‑on lead role. You have a deep understanding of data architecture and engineering best practices, and extensive experience designing and building cloud data platforms and ETL pipelines that are scalable, efficient, robust and extensible.

You’ll need significant recent hands‑on experience using AI tools and techniques to accelerate data architecture and engineering, including technologies such as Claude Code, Codex, Amazon Bedrock or similar.

Your experience should ideally cover the following areas and technologies:

• SQL, dbt, dlt, and data modelling
• Python, Git, Docker
• ETL pipelines and data platforms
• Relational database engines, including PostgreSQL and DuckDB
• Orchestration (e.g. Step Functions, Airflow)
• AI-accelerated development workflows and tools including Claude Code and Codex
• AWS cloud infrastructure, including Lambda and ECS
• End-to-end data pipeline testing strategy and implementation
• Vendor review and tooling selection
• Data warehousing
• IaC with CDK
• CI/CD pipelines

Benefits

Join a friendly, fun, and focused team working at the intersection of technology and the real world. Every day poses a novel technical challenge where you’ll have the opportunity to have real impact by driving technical architecture and project direction.

• Interesting technical challenges with real impact.
• Shape technical decisions and project direction.
• Work with experts in their fields.
• 25 days’ annual leave, plus bank holidays and your birthday off!
• A new MacBook Pro on joining us, yours to keep if you're with us for more than 6 months.
• Competitive salary or day rate.
• London hybrid, usually 1–2 days a week together.

This role is open to permanent or contract candidates. Please note some benefits apply only to permanent employees.

DataTruth’s Culture

Compassion: Empathetic Engagement

We take time to understand each other, listen to concerns and offer support. You’ll work with people who treat each other with kindness and are willing to help.

Impact: Purposeful Action

We care about the difference our work makes. We take responsibility for our decisions and consider their long‑term effects on the people they reach.

Optimism: Positive Mindset

We approach challenges with a positive outlook and a willingness to learn. We support each other through difficulties and look for a constructive way forward.

Respect: Human‑Centred Partnership

We value different perspectives and experiences, treat people with respect and make room for everyone to contribute. We want everyone to feel included and part of the team.