Principal AWS Data Engineer: Lead Pipelines (Hybrid)

Capco

Greater London

Hybrid

GBP 120,000 - 180,000

Full time

24 hours ago
Be an early applicant

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Benefits offered by this job

Discretionary bonus
Competitive pension
Health insurance
Life insurance
Critical illness cover
Mental health support
Family-friendly leave
Backup care

Job summary

Capco in London is seeking a Principal Data Engineer with AWS expertise to design, build and deploy scalable data pipelines for Tier 1 financial services clients. You will own technical direction, influence architecture and ensure secure, reliable data platforms across greenfield and enterprise-scale programmes.

You will mentor engineers, promote engineering excellence, and work across teams to balance performance, security and long-term maintainability while leveraging AI-assisted tooling under

Qualifications

  • Strong proficiency in at least one OOP language such as Python, Scala, or Java.
  • Hands-on experience of Big Data technologies including Spark, Hadoop, or similar distributed processing frameworks.
  • Experience building real-time event streaming solutions using technologies such as Kafka, Spark Streaming, or Amazon Kinesis within AWS environments.
  • Proven expertise designing modern data architectures including Data Lakehouse and Data Warehousing solutions, alongside CI/CD, DevOps practices, and data governance including GDPR.

Responsibilities

  • Own the technical design and delivery of end-to-end data pipelines using AWS-native services and modern cloud data architectures, ensuring scalable, secure, resilient and maintainable solutions.
  • Partner with clients and cross-functional teams to understand business requirements, challenge assumptions, evaluate architectural trade-offs and translate complex challenges into production-grade data platforms that deliver measurable value.
  • Apply AWS Well-Architected Principles, engineering best practices and CI/CD approaches to improve quality, efficiency and operational excellence. Use approved AI-assisted development tools and automation, where appropriate, to accelerate activities such as test generation, code review preparation and technical documentation, while maintaining human validation, security controls and clear escalation of potential risks.
  • Lead the development of robust, tested and fault-tolerant data engineering solutions, taking accountability for technical quality, operational reliability and sustainable engineering decisions, while promoting appropriate governance standards.
  • Mentor engineers, share technical expertise and establish a culture of engineering excellence through constructive technical challenge, knowledge sharing and continuous improvement.

Skills

Python
Scala
Java
Spark
Hadoop
Kafka
Spark Streaming
Kinesis
Data Lakehouse
Data Warehousing
CI/CD
DevOps
GDPR
Data governance

Tools

Databricks
AWS
CI/CD pipelines

Job description

Capco in London is seeking a Principal Data Engineer with AWS expertise to design, build and deploy scalable data pipelines for Tier 1 financial services clients. You will own technical direction, influence architecture and ensure secure, reliable data platforms across greenfield and enterprise-scale programmes.

You will mentor engineers, promote engineering excellence, and work across teams to balance performance, security and long-term maintainability while leveraging AI-assisted tooling under

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Senior Data Consultant: Secure AWS Pipelines Lead (Hybrid)
Senior Data Consultant: Secure AWS Pipelines Lead (Hybrid)

Cyber Security training courses • Greater London

Hybrid
GBP 83,000 - 109,000
AWS Data Engineer — Hybrid London, Build Scalable Pipelines
AWS Data Engineer — Hybrid London, Build Scalable Pipelines

Datatech Analytics • Greater London

Hybrid
GBP 45,000 - 75,000
Senior Data Engineer: AWS Pipelines Lead & Mentor (Hybrid)
Senior Data Engineer: AWS Pipelines Lead & Mentor (Hybrid)

Center Parcs • Newark on Trent

Hybrid
GBP 60,000 - 65,000
Bonus scheme
Pension & Life Assurance
Free Day Passes
+5
Senior Data Engineer - AWS
Senior Data Engineer - AWS

Capco • Greater London

Hybrid
GBP 90,000 - 120,000
Competitive pension
Health insurance
Discretionary bonus
+5
Senior Data Engineer: AWS Pipelines, Lead & Hybrid Role
Senior Data Engineer: AWS Pipelines, Lead & Hybrid Role

Center Parcs • Newark and Sherwood

Hybrid
GBP 60,000 - 65,000
Bonus scheme
Pension
Recognition & rewards
+6
London Hybrid Senior Data Engineer: Data Pipelines & Impact
London Hybrid Senior Data Engineer: Data Pipelines & Impact

EPAM Systems • Greater London

Hybrid
GBP 90,000 - 150,000
ESPP
Private medical insurance
Dental care
+3
Senior AWS DevOps Engineer – Hybrid London, CI/CD & Data
Senior AWS DevOps Engineer – Hybrid London, CI/CD & Data

Expert Employment Limited • City Of London

Hybrid
GBP 90,000 - 130,000
Hybrid working
Senior Data Engineer | Hybrid London | Lead Pipelines
Senior Data Engineer | Hybrid London | Lead Pipelines

dcoded. | B Corp ™ • Greater London

Hybrid
GBP 60,000 - 75,000
Hybrid working with flexible remote
9am–6pm working hours
Principal Data Engineer - AWS
Principal Data Engineer - AWS

Capco • Greater London

Hybrid
GBP 120,000 - 180,000
Discretionary bonus
Competitive pension
Health insurance
+5
Contract AWS Data Engineer: ETL Pipelines & Analytics
Contract AWS Data Engineer: ETL Pipelines & Analytics

WRK Digital • United Kingdom

On-site
GBP 70,000 - 90,000