Senior AWS Data Engineer - SC Cleared

AgileSource Ltd; England; On-site; GBP 85,000 - 125,000; Full time

Job summary

AgileSource Ltd. seeks a Senior AWS Data Engineer to lead design and build of modern data platforms on AWS. You will work with architects, security specialists and client teams to create robust, secure data solutions.

Role involves hands-on engineering, establishing standards, and mentoring colleagues. Requires AWS expertise, Python/SQL, and a focus on governance and secure delivery.

Qualifications

• Significant hands-on data engineering in enterprise environments.
• Strong experience with AWS data services and architectures.
• Proficient in Python and SQL for data tooling and pipelines.
• Knowledge of data modelling, data lakes/warehouses and modern platforms.
• Experience with infrastructure-as-code (Terraform) and CI/CD for data workloads.
• Familiarity with data security, governance and compliance requirements.

Responsibilities

• Design, build and maintain scalable data platforms and pipelines on AWS.
• Develop robust ETL/ELT processes to ingest/transform data from multiple sources.
• Implement data solutions using AWS services (S3, Glue, Lambda, Athena, Redshift, EMR).
• Create data models and storage strategies for analytic/operational needs.
• Code in Python/SQL and apply modern data engineering frameworks.
• Apply IaC and automation to deploy and manage data platforms.
• Build data quality, monitoring and observability into solutions.
• Ensure security, governance and data protection throughout design/delivery.
• Collaborate with architects and security teams to integrate solutions.
• Engage with clients/stakeholders to explain decisions and recommendations.
• Troubleshoot data, performance, and integration issues across environments.
• Develop and promote data engineering standards and best practices.
• Mentor engineers and foster knowledge sharing and continuous improvement.

Skills

AWS

Python

SQL

ETL/ELT

Data Modeling

Terraform

Data Quality

Job description

AgileSoruce are seeking experienced Senior AWS Data Engineers for Permanent or Contract opportunities.

Permanent or Contract (Inside IR35)

Must have Live SC Clearance

As a Senior Data Engineer, you’ll take a leading role in designing and building modern data platforms, pipelines and services within AWS.

You’ll work closely with architects, engineers, security specialists and client teams to understand requirements and turn them into robust, maintainable and secure data solutions.

Alongside hands‑on engineering, you’ll help establish engineering standards and patterns, contribute to key technical decisions and support the development of other engineers within the team.

What You’ll Do

• Design, build and maintain scalable data platforms and data pipelines within AWS
• Develop robust ETL/ELT processes for ingesting, transforming and processing data from multiple sources
• Design and implement data solutions using AWS services such as S3, Glue, Lambda, Athena, Redshift, EMR and related technologies
• Design data models and storage approaches appropriate to analytical and operational requirements
• Develop solutions using Python, SQL and modern data engineering frameworks
• Apply infrastructure-as-code and automation approaches to the deployment and management of data platforms
• Build data quality, validation, monitoring and observability into engineering solutions
• Ensure security, governance and data protection requirements are considered throughout solution design and delivery
• Work with architects, cloud engineers, security specialists and other SMEs to ensure solutions integrate effectively across the wider technology landscape
• Engage with clients and technical stakeholders to understand requirements, explain technical decisions and provide recommendations
• Troubleshoot complex data, performance and integration issues across development and production environments
• Define and promote data engineering standards, reusable patterns and best practices
• Mentor and support other engineers, encouraging knowledge sharing and continuous improvement

What We’re Looking For

• Significant hands‑on experience as a Data Engineer, ideally within complex enterprise environments
• Strong experience designing and delivering data solutions on AWS
• Strong knowledge of AWS data services such as S3, Glue, Lambda, Athena, Redshift and EMR, with experience selecting the appropriate services for different requirements
• Strong Python and SQL skills
• Experience designing and building scalable ETL/ELT pipelines and data processing workflows
• Good understanding of data modelling, data lakes, data warehouses and modern data platform architectures
• Experience with infrastructure-as-code, ideally Terraform
• Experience with CI/CD and automated testing for data engineering workloads
• Good understanding of data security, governance, access control and data quality
• Experience making technical design decisions and providing engineering leadership within multidisciplinary teams
• Strong troubleshooting and problem‑solving skills
• Confidence working directly with clients and communicating technical concepts, decisions and recommendations
• Experience supporting and mentoring other engineers
• Current SC clearance is required due to the nature and timescales of our client engagements.
• AWS certification at Associate or Professional level
• Experience with streaming and event‑driven data technologies
• Experience with technologies such as Spark, Kafka or Databricks
• Experience delivering data platforms within public sector, defence or other regulated environments
• Experience working within secure environments and understanding associated security and assurance requirements
• Exposure to machine learning or AI workloads and the data platforms that support them