SC Clearance AWS Data Engineer — Cloud Data Pipelines

UBDS Group; Manchester; On-site; GBP 80,000 - 105,000; Full time

Job summary

UBDS Group is seeking an experienced AWS Data Engineer to join our Manchester consultancy. The role focuses on designing, building and maintaining scalable data pipelines and infrastructure using AWS-native services.

You will work across data orchestration, automation and containerisation, delivering secure, high-performance data solutions for clients with active SC clearance requirements. This client-facing position is hands-on and requires strong problem solving.

Qualifications

• Strong hands-on experience in AWS Data Engineering.
• Excellent Python programming skills and strong SQL proficiency.
• Experience using Apache Airflow for workflow orchestration and DAG management.
• Practical experience with Docker and containerisation.
• Strong knowledge of AWS data services (Redshift, Glue, Lambda, Step Functions, Batch, OpenSearch).
• Experience with CI/CD, Git and YAML configurations.
• Good understanding of AWS security, IAM RBAC.

Responsibilities

• Design, develop and maintain scalable data pipelines using Python, SQL and Airflow.
• Build and optimise cloud-based data solutions using AWS services (Redshift, Glue, Lambda, Step Functions, Batch, OpenSearch).
• Develop and manage containerised applications and workflows with Docker.
• Implement and maintain data orchestration processes for reliable data processing.
• Collaborate with engineering teams and stakeholders to deliver secure, scalable data infrastructure.
• Monitor, troubleshoot and optimise data pipeline performance.
• Work with GitLab or similar tools for version control and CI/CD.
• Ensure data quality, governance and security best practices are followed.
• Contribute to technical documentation, automation and CI/CD improvements.

Skills

AWS Data Eng

Python

SQL

Airflow

Docker

CI/CD

Git

YAML

Security RBAC

Agile

Tools

Terraform

Confluence

GitLab

Job description

UBDS Group is seeking an experienced AWS Data Engineer to join our Manchester consultancy. The role focuses on designing, building and maintaining scalable data pipelines and infrastructure using AWS-native services.

You will work across data orchestration, automation and containerisation, delivering secure, high-performance data solutions for clients with active SC clearance requirements. This client-facing position is hands-on and requires strong problem solving.