Lead Data Engineer

Experis; City Of London; On-site; GBP 70,000 - 90,000; Full time

Job summary

A leading technology solutions provider is seeking a Lead Data Engineer to develop scalable data solutions and support AI initiatives in a hybrid working environment. You will design data pipelines, work with cloud technology, and ensure effective integration across teams. This position offers an opportunity to be involved in transformative projects that have a significant impact on the future of industries.

Qualifications

• Strong experience in data pipeline development and orchestration.
• Proficient with cloud platforms (Snowflake, AWS fundamentals).
• Solid understanding of data architecture, warehousing, and modelling.

Responsibilities

• Build and orchestrate data pipelines across Snowflake and AWS.
• Develop pipeline programming using Python, Spark, and SQL.
• Implement MLOps, CI/CD, and data quality frameworks.

Skills

Data pipeline development

Cloud platforms (Snowflake, AWS)

Data architecture and modelling

Programming (Python, Spark, SQL)

ML/AI frameworks knowledge

CI/CD and data testing

Collaboration in multi-team environments

Job description

Job Title: Lead Data Engineer

Location: London (Hybrid)

Contract: 6 Months (Potential Extension)

Start Date: ASAP

About the Client

Our client is transforming their industry by replacing cigarettes with innovative, smoke‑free alternatives. They are leveraging technology, data, and AI to drive a global shift toward a smoke‑free world. This is a fast‑paced, high‑impact environment, perfect for candidates who are strategic, independent, and excited to work at the forefront of data and AI innovation.

The Role

We are looking for a skilled Data Engineer to design, build, and optimise enterprise‑scale data pipelines and cloud platforms. You will translate business and AI/ML requirements into robust, scalable solutions while collaborating across multi‑disciplinary teams and external vendors.

As a key member of the data architecture you will:

• Build and orchestrate data pipelines across Snowflake and AWS environments.
• Apply data modelling, warehousing, and architecture principles (Kimball/Inmon).
• Develop pipeline programming using Python, Spark, and SQL; integrate APIs for seamless workflows.
• Support Machine Learning and AI initiatives, including NLP, Computer Vision, Time Series, and LLMs.
• Implement MLOps, CI/CD pipelines, data testing, and quality frameworks.
• Act as an AI super‑user, applying prompt engineering and creating AI artifacts.
• Work independently while providing clear justification for technical decisions.

Key Skills & Experience

• Strong experience in data pipeline development and orchestration.
• Proficient with cloud platforms (Snowflake, AWS fundamentals).
• Solid understanding of data architecture, warehousing, and modelling.
• Programming expertise: Python, Spark, SQL, API integration.
• Knowledge of ML/AI frameworks, MLOps, and advanced analytics concepts.
• Experience with CI/CD, data testing frameworks, and versioning strategies.
• Ability to work effectively in multi‑team, vendor‑integrated environments.

Why This Role

• Join a global, transformative initiative shaping a smoke‑free future.
• Work with cutting‑edge cloud, AI, and data technologies.
• Opportunity to influence technical and strategic decisions across enterprise data delivery.
• Dynamic, innovative environment where your work has real business impact.