Lead Data Engineer: AWS, DataOps & Platform Migration

AND Digital; Greater London; Hybrid; GBP 70,000 - 90,000; Full time

Job summary

A digital technology specialist is seeking a Lead Data Engineer to guide a significant migration from Regshift to AWS RDS. The role combines technical leadership with mentorship to transition Data Analysts through new Python practices, ensure quality in data architecture, and optimize performance. With a focus on collaboration and skill development, the position promises a dynamic work environment that challenges candidates while fostering continuous learning and growth.

Benefits

25 days holiday plus bank holidays

Share scheme

£1000 flexifund for personalized benefits

6% pension

Annual training budget

Qualifications

• 7 to 10 years experience in Data Engineering with full-stack data pipelines.
• 5+ years experience with AWS, ideally RDS and Redshift administration.
• 5+ years of expert-level Python for ETL pipeline development.
• 5+ years of SQL for OLTP and OLAP PostgreSQL database architecture.
• 3+ years of Git expertise and schema management tools.

Responsibilities

• Lead Data Pipeline Delivery by implementing new Python-based ETL pipelines.
• Optimize new Postgres OLTP for transactional performance.
• Establish and manage DataOps standards including version control and CI/CD.
• Mentor team members on Python, Git, and engineering workflows.
• Network with business domains to capture data requirements.

Skills

Data Engineering

AWS

Python

PostgreSQL

DataOps / Tools

Team Leadership

Tools

Git

Airflow

Pandas

Job description

A digital technology specialist is seeking a Lead Data Engineer to guide a significant migration from Regshift to AWS RDS. The role combines technical leadership with mentorship to transition Data Analysts through new Python practices, ensure quality in data architecture, and optimize performance. With a focus on collaboration and skill development, the position promises a dynamic work environment that challenges candidates while fostering continuous learning and growth.