Senior Data Engineer - Banking, Machine Learning

Harvey Nash; London; On-site; GBP 60,000 - 90,000; Full time

Job summary

A leading investment bank in London is seeking a Senior Data Engineer specializing in Machine Learning and complex data infrastructure. The role involves designing scalable data pipelines and optimizing workflows, requiring extensive experience in data engineering and cloud technologies.

Qualifications

• 5+ years of experience in data engineering.
• Strong knowledge of Spark and Hadoop ecosystems.
• Proficiency in Python, Scala, or Java.

Responsibilities

• Design and implement data pipelines for large data volumes.
• Architect end-to-end solutions supporting ML workflows.
• Mentor junior engineers in data engineering best practices.

Skills

Data Pipeline Design

Cloud Data Platforms

Machine Learning Integration

Distributed Computing

Data Quality and Governance

Education

Bachelor's degree in Computer Science

Master's preferred

Tools

AWS

Azure

Spark

Hadoop

Airflow

Job description

Senior Data Engineer - Banking, Machine Learning sought by leading investment bank based in the city of London. Inside IR35 - 3 days a week on site Job Overview
We are seeking an experienced Data Engineer to design, build, and optimize our complex data pipelines and big data infrastructure. The ideal candidate will have deep expertise in architecting scalable data solutions, integrating machine learning workflows, and optimizing data processing systems. This role requires both technical excellence and the ability to influence architectural decisions that align with business objectives.
Key Responsibilities

• Design and implement robust, scalable data pipelines for processing large volumes of structured and unstructured data
• Architect end-to-end data solutions that support machine learning model training and deployment
• Develop and maintain data infrastructure that ensures data quality, reliability, and accessibility
• Optimize existing data workflows for performance, cost-efficiency, and maintainability
• Contribute to strategic architectural decisions and technical roadmaps
• Implement data governance and security best practices across the data ecosystem
• Mentor junior engineers and promote best practices in data engineering

Required Skills & Qualifications

• 5+ years of experience in data engineering roles with progressively increasing responsibility
• Proven experience designing and implementing complex data pipelines at scale
• Strong knowledge of distributed computing frameworks (Spark, Hadoop ecosystem)
• Experience with cloud-based data platforms (AWS, Azure, GCP)
• Proficiency in data orchestration tools (Airflow, Prefect, Dagster, or similar)
• Solid programming skills in Python, Scala, or Java
• Experience integrating ML workflows into production data systems
• Strong understanding of data modeling, ETL processes, and database design
• Demonstrated ability to architect solutions for big data challenges

Preferred Qualifications

• Experience with real-time data processing (Kafka, Kinesis, Flink)
• Knowledge of containerization and infrastructure-as-code (Docker, Kubernetes, Terraform)
• Familiarity with MLOps practices and tools (MLflow, Kubeflow, etc.)
• Experience with data governance frameworks and data cataloging
• Understanding of graph databases and unstructured data processing
• Knowledge of advanced analytics techniques and statistical methods
• Experience with data mesh or data fabric architectural patterns

Education

• Bachelor's degree in Computer Science, Data Science, or related field (Master's preferred)
• Relevant certifications in cloud platforms or data technologies

Please apply within for further details or call on

Alex Reeder

Harvey Nash Finance & Banking

Create a job alert and receive personalised job recommendations straight to your inbox.