Senior Data Engineer - AWS

Capco; Greater London; On-site; GBP 90,000 - 120,000; Full time

Benefits offered by this job

Competitive pension

Health insurance

Discretionary bonus

Continuous learning opportunities

Gym membership

Travel insurance

Family care support

Cycle to work

Job summary

A leading consulting firm is seeking a Senior Data Engineer to design and build AWS-based data pipelines in a hybrid model. The role involves collaborating with clients to gather requirements and applying AWS principles for scalable solutions. Ideal candidates should have strong skills in Python, Scala, or Java, and experience with Big Data technologies like Spark and Hadoop. This opportunity includes comprehensive benefits and a commitment to inclusivity and professional growth.

Qualifications

• Proficient in Python, Scala, or Java with Big Data experience.
• Practical knowledge of real-time event streaming pipelines.
• Proficiency in AWS cloud environments.
• Experience in modern data architectures including Data Lakehouse.

Responsibilities

• Design and build end-to-end data pipelines leveraging AWS.
• Collaborate with clients for requirements and define solutions.
• Apply AWS Well-Architected Principles for scalability.
• Lead development and mentor junior engineers.
• Support and share knowledge across the team.

Skills

Python

Scala

Java

Big Data technologies

AWS

Data Engineering best practices

Tools

Spark

Hadoop

Kafka

Kinesis

Databricks

Airflow

Job description

Overview

Senior Data Engineer - AWS
Location: London (Hybrid) | Practice Area: Technology & Engineering | Type: Permanent

Engineer future-ready data platforms that power financial transformation.

The Role

We’re looking for a Senior Data Engineer with AWS to join our growing team of engineering experts driving next-gen transformation for Tier 1 financial services clients. You’ll play a pivotal role in designing, building, and deploying cloud-based data pipelines. Working across greenfield projects and enterprise-scale platforms, your work will directly impact the way data is ingested, transformed, and served at scale across the financial services industry.

What You’ll Do

• Design and build end-to-end data pipelines leveraging AWS-native tools and modern data architectures.
• Collaborate with clients to gather requirements, define solutions, and deliver production-grade systems.
• Apply AWS Well-Architected Principles to ensure scalability, security, and resilience.
• Lead in the development of robust, tested, and fault-tolerant data engineering solutions.
• Support and mentor junior engineers, contributing to knowledge sharing across the team.

What We’re Looking For

• Proficient in one of Python, Scala or Java, with strong experience in Big Data technologies such as Spark, Hadoop etc.
• Practical knowledge of building real-time event streaming pipelines (e.g., Kafka, Spark Streaming, Kinesis).
• Proficiency in AWS cloud environments.
• Proven experience developing modern data architectures including Data Lakehouse and Data Warehousing.

A solid understanding of CI/CD practices, DevOps tooling, and data governance including GDPR.

Bonus Points For

• Expertise in Data Modelling, schema design, and handling both structured and semi-structured data.
• Familiarity with distributed systems such as Hadoop, Spark, HDFS, Hive, Databricks.
• Exposure to AWS Lake Formation and automation of ingestion and transformation layers.
• Background in delivering solutions for highly regulated industries.
• Passion for mentoring and enabling data engineering best practices across teams.

Why Join Capco

• Deliver high-impact technology solutions for Tier 1 financial institutions
• Work in a collaborative, flat, and entrepreneurial consulting culture
• Access continuous learning, training, and industry certifications
• Be part of a team shaping the future of digital financial services
• Help shape the future of digital transformation across FS & Energy

Benefits

• Core Benefits: Discretionary bonus, competitive pension, health insurance, life insurance and critical illness cover.
• Mental Health: Easy access to CareFirst, Unmind, Aviva consultations, and in-house first aiders.
• Family-Friendly: Maternity, adoption, shared parental leave, plus paid leave for sickness, pregnancy loss, fertility treatment, menopause, and bereavement.
• Family Care: 8 complimentary backup care sessions for emergency childcare or elder care.
• Holiday Flexibility: 5 weeks of annual leave with the option to buy or sell holiday days based on your needs.
• Continuous Learning: Minimum 40 hours of training annually and a Business Coach from Day One.
• Extra Perks: Gympass, travel insurance, Tastecard, season ticket loans, Cycle to Work, and dental insurance.

Inclusion at Capco

We’re committed to a barrier-free, inclusive recruitment process. If you need any adjustments at any stage, just let us know – we’ll be happy to help. We welcome applicants from all backgrounds. At Capco, we value the difference you make, and the differences that make you. Our #BeYourselfAtWork culture champions diversity, equity and inclusivity, and we bring a collaborative mindset to our partnerships with clients and colleagues. #BeYourselfAtWork is the cornerstone of our success and a value that our employees live and breathe every day. #LI-DNI