Engineering Manager - AWS Data & Lakehouse Pipelines

London Stock Exchange Group; Greater London; On-site; GBP 40,000 - 60,000; Full time

Job summary

A leading global financial markets infrastructure provider based in the UK is looking for a Software Data Engineer to join their team. You will design and develop scalable data pipelines using Python and Apache Spark within a modern AWS environment. The ideal candidate will have experience in data engineering, a solid understanding of AWS tools, and a passion for high-quality data solutions. If you're driven by innovation and impact, this role offers a great opportunity for professional growth.

Qualifications

• Proficient in writing clean, maintainable Python code.
• Understanding of batch processing and schema evolution for ETL.
• Eager to learn about data lakehouse architecture.

Responsibilities

• Design and develop scalable data pipelines using Python and Apache Spark.
• Orchestrate data workflows with AWS tools like Glue and Lambda.
• Participate in code reviews and architecture discussions.

Skills

Python coding

Data engineering basics

Apache Spark

AWS data stack

Tools

Apache Spark

AWS (S3, Glue, Lambda, EMR)

CI/CD tools (GitLab CI, Jenkins, GitHub Actions)

Job description

A leading global financial markets infrastructure provider based in the UK is looking for a Software Data Engineer to join their team. You will design and develop scalable data pipelines using Python and Apache Spark within a modern AWS environment. The ideal candidate will have experience in data engineering, a solid understanding of AWS tools, and a passion for high-quality data solutions. If you're driven by innovation and impact, this role offers a great opportunity for professional growth.