Data Engineer – Advanced Analytics & AI Pipelines

IBM; City of Westminster; Hybrid; GBP 80,000 - 130,000; Full time

Benefits offered by this job

Flexible working

Sabbatical programs

Paternity leave

Maternity leave

Maternity returners scheme

25 days holiday

Private medical

Dental & optical cover

Employee Assistance Program

Life assurance

Group pension plan

Job summary

IBM Consulting UK FutureNow is seeking a skilled Data Engineer to design and implement scalable data pipelines, enabling analytics, ML, and AI workloads across enterprise environments. You'll partner with Data Scientists, AI Engineers, Architects and stakeholders to deliver modern data products within Hybrid Cloud & Data, applying best practices in data governance, quality, and security.

The role emphasizes mentoring junior engineers, contributing to platform design choices, and staying current

Qualifications

• Strong experience designing and building data engineering solutions within enterprise environments.
• Proficiency in Python and SQL.
• Experience with Pandas, NumPy, PySpark, and Dask.
• Experience developing ETL/ELT pipelines and data transformation solutions.
• Exposure to analytics, machine learning, or AI-enabled data workloads.
• Experience with cloud-based analytics and data platforms.
• Strong understanding of data modelling, data quality, and data governance principles.
• Excellent communication and stakeholder management skills.
• Experience working within Agile delivery environments.

Responsibilities

• Design, build, and optimise data pipelines supporting analytics, machine learning, and AI workloads.
• Develop scalable data solutions across structured and unstructured data environments.
• Engineer datasets and analytical products that enable Data Science, AI, reporting, and operational decision-making.
• Build and maintain data processing frameworks, transformation pipelines, and integration patterns.
• Support advanced analytics, predictive modelling, and AI initiatives through high-quality data engineering.
• Collaborate with Architects, Data Scientists, AI Engineers, and business stakeholders throughout the delivery lifecycle.
• Translate business requirements into scalable and maintainable technical solutions.
• Contribute to modern data platform design, data governance, and engineering best practices.
• Mentor junior practitioners and help develop team capability.
• Continuously explore emerging technologies and approaches across Data Engineering, Analytics, and AI.

Skills

Python

SQL

Pandas

NumPy

PySpark

Dask

ETL/ELT pipelines

Cloud platforms

Data modeling

Data governance

Data quality

Agile

Stakeholder management

Communication

Spark

Databricks

Snowflake

Synapse

Microsoft Fabric

ML/AI workloads

Public Sector / regulated environments

Mentoring

Distributed data processing

Tools

Databricks

Microsoft Fabric

Snowflake

Synapse

Spark

Job description

IBM Consulting UK FutureNow is seeking a skilled Data Engineer to design and implement scalable data pipelines, enabling analytics, ML, and AI workloads across enterprise environments. You'll partner with Data Scientists, AI Engineers, Architects and stakeholders to deliver modern data products within Hybrid Cloud & Data, applying best practices in data governance, quality, and security.

The role emphasizes mentoring junior engineers, contributing to platform design choices, and staying current