Senior AWS Data Engineer

UBDS Group; Greater London; On-site; GBP 70,000 - 110,000; Full time

Benefits offered by this job

Private medical cover

Discretionary bonus

Holiday – 25 days + birthday + work-周年

EV leasing

Pension scheme

Death in service cover

Job summary

UBDS Group seeks an experienced Senior Data Engineer to design, build and deliver scalable data platforms within AWS. You’ll mentor engineers, guide patterns, and collaborate with architects and security professionals to ensure secure, resilient data solutions for clients in regulated environments.

You’ll lead data ingestion, transformation and governance efforts, leveraging S3/Glue/Lambda/Redshift and related AWS services, while applying CI/CD and IaC practices to deployments.

Qualifications

• Significant hands‑on experience as a Data Engineer, ideally within complex enterprise environments
• Strong experience designing and delivering data solutions on AWS
• Strong Python and SQL skills
• Experience with infrastructure‑as‑code, ideally Terraform
• Experience with CI/CD and automated testing for data engineering workloads
• Good understanding of data security, governance, access control and data quality
• Current SC clearance is highly desirable due to the nature and timescales of our client engagements

Responsibilities

• Design, build and maintain scalable data platforms and data pipelines within AWS
• Develop robust ETL/ELT processes for ingesting, transforming and processing data from multiple sources
• Design and implement data solutions using AWS services such as S3, Glue, Lambda, Athena, Redshift, EMR and related technologies
• Design data models and storage approaches appropriate to analytical and operational requirements
• Develop solutions using Python, SQL and modern data engineering frameworks
• Apply infrastructure‑as‑code and automation approaches to the deployment and management of data platforms
• Build data quality, validation, monitoring and observability into engineering solutions
• Ensure security, governance and data protection requirements are considered throughout solution design and delivery
• Work with architects, cloud engineers, security specialists and other SMEs to ensure solutions integrate effectively across the wider technology landscape
• Engage with clients and technical stakeholders to understand requirements, explain technical decisions and provide recommendations
• Troubleshoot complex data, performance and integration issues across development and production environments
• Define and promote data engineering standards, reusable patterns and best practices
• Mentor and support other engineers, encouraging knowledge sharing and continuous improvement

Skills

Python

SQL

Data modeling

CI/CD

Data governance

Security

Tools

Terraform

AWS S3

AWS Glue

AWS Lambda

AWS Athena

AWS Redshift

AWS EMR

Job description

About Rayo

Rayo, part of UBDS Group, helps organisations solve complex technology challenges across cloud, data, AI and digital transformation.

We work with clients operating in complex, secure and highly regulated environments, with a significant focus on the public sector and defence.

We’re looking for an experienced Senior Data Engineer with strong AWS expertise to design, build and deliver scalable, secure and resilient data solutions for our clients.

This is a hands‑on senior engineering role, combining deep data engineering expertise with technical ownership, client engagement and the ability to guide other engineers and influence engineering practices.

The Role

As a Senior Data Engineer, you’ll take a leading role in designing and building modern data platforms, pipelines and services within AWS.

You’ll work closely with architects, engineers, security specialists and client teams to understand requirements and turn them into robust, maintainable and secure data solutions.

Alongside hands‑on engineering, you’ll help establish engineering standards and patterns, contribute to key technical decisions and support the development of other engineers within the team.

What You’ll Do

• Design, build and maintain scalable data platforms and data pipelines within AWS
• Develop robust ETL/ELT processes for ingesting, transforming and processing data from multiple sources
• Design and implement data solutions using AWS services such as S3, Glue, Lambda, Athena, Redshift, EMR and related technologies
• Design data models and storage approaches appropriate to analytical and operational requirements
• Develop solutions using Python, SQL and modern data engineering frameworks
• Apply infrastructure‑as‑code and automation approaches to the deployment and management of data platforms
• Build data quality, validation, monitoring and observability into engineering solutions
• Ensure security, governance and data protection requirements are considered throughout solution design and delivery
• Work with architects, cloud engineers, security specialists and other SMEs to ensure solutions integrate effectively across the wider technology landscape
• Engage with clients and technical stakeholders to understand requirements, explain technical decisions and provide recommendations
• Troubleshoot complex data, performance and integration issues across development and production environments
• Define and promote data engineering standards, reusable patterns and best practices
• Mentor and support other engineers, encouraging knowledge sharing and continuous improvement

What We’re Looking For

• Significant hands‑on experience as a Data Engineer, ideally within complex enterprise environments
• Strong experience designing and delivering data solutions on AWS
• Strong knowledge of AWS data services such as S3, Glue, Lambda, Athena, Redshift and EMR, with experience selecting the appropriate services for different requirements
• Strong Python and SQL skills
• Experience designing and building scalable ETL/ELT pipelines and data processing workflows
• Good understanding of data modelling, data lakes, data warehouses and modern data platform architectures
• Experience with infrastructure‑as‑code, ideally Terraform
• Experience with CI/CD and automated testing for data engineering workloads
• Good understanding of data security, governance, access control and data quality
• Experience making technical design decisions and providing engineering leadership within multidisciplinary teams
• Strong troubleshooting and problem‑solving skills
• Confidence working directly with clients and communicating technical concepts, decisions and recommendations
• Experience supporting and mentoring other engineers
• Current SC clearance is highly desirable due to the nature and timescales of our client engagements. Candidates must be eligible to obtain and maintain SC clearance.

Desirable

• AWS certification at Associate or Professional level
• Experience with streaming and event‑driven data technologies
• Experience with technologies such as Spark, Kafka or Databricks
• Experience delivering data platforms within public sector, defence or other regulated environments
• Experience working within secure environments and understanding associated security and assurance requirements
• Exposure to machine learning or AI workloads and the data platforms that support them

Why people choose to grow their careers at UBDS Group

Professionals choose to grow their careers at UBDS Group for its reputation as a dynamic and forward‑thinking organisation that is deeply committed to both innovation and employee development. At UBDS Group, employees are given unique opportunities to work on cutting‑edge projects across a diverse range of industries, exposing them to new challenges and learning opportunities that are pivotal for professional growth. The Group’s culture emphasises continuous improvement, offering ample training programs, mentorship, and the chance to gain certifications that enhance their skills and marketability.

UBDS Group fosters a collaborative environment where creativity and innovation are encouraged, allowing employees to contribute ideas and solutions that have a tangible impact on the company and its clients. This combination of professional development, a culture of innovation, and the opportunity to make meaningful contributions makes UBDS Group an attractive place for those looking to advance their careers and be at the forefront of technological and operational excellence.

Employee Benefits

• Training – All team members are offered a number of options in terms of personal development, whether it is technical led, business acumen or methodologies. We want you to grow with us and to help us achieve more
• Private medical cover for you and your spouse/partner, offered via Vitality
• Discretionary bonus based on a blend of personal and company performance
• Holiday – You will receive 25 Days holiday, plus 1 day for Birthday and 1 day for your work anniversary in addition to UK bank holidays
• Electric Vehicle leasing with salary sacrifice
• Contributed Pension Scheme
• Death in service cover

About UBDS Group

At UBDS Group our mission is to support entrepreneurs who are setting new standards with technology solutions across cloud services, cybersecurity, data and AI, ensuring that every investment advances our commitment to innovation, making a difference, and creating impactful solutions for organisations and society.

Equal Opportunities

We are an equal opportunities employer and do not discriminate on the grounds of gender, sexual orientation, marital or civil partner status, pregnancy or maternity, gender reassignment, race, colour, nationality, ethnic or national origin, religion or belief, disability or age.