About the Role
As a Data Engineer on the Data Services & Operations team, you will be the primary point of contact for data requests - connecting business users with the right data assets while upholding governance, security, and service standards. The role is equal parts engineering, stakeholder partnership, and operational excellence.
Key Responsibilities
Data Pipeline & Architecture Design
- Design and implement scalable ETL/ELT pipelines using AWS services including Glue, Step Functions, Lambda and S3.
- Architect and build data lakehouse solutions leveraging Apache Iceberg or S3 tables - incorporating schema evolution, partition evolution and ACID transactions.
- Optimise pipelines for performance, cost and reliability at scale.
Data Quality & Governance
- Define, implement and maintain automated data quality validation frameworks, with metrics and monitoring to ensure data accuracy, completeness and consistency.
- Enforce data governance standards and ensure compliance across the data platform.
Application Development & Deployment
- Write production-quality code and deploy solutions on AWS cloud infrastructure.
- Apply DevOps practices - CI/CD pipelines and infrastructure-as-code tools such as Terraform or CloudFormation - for repeatable, auditable deployments.
Collaboration & Handover
- Work closely with cross-functional teams and communicate technical designs clearly to non-technical stakeholders.
- Produce thorough technical documentation to support the handover of solutions to the Day 2 operations team.
Qualifications
- Degree in Computer Science, Data Engineering, Information Systems or a related field.
- 3-5 years of experience in data engineering, ETL/ELT development or data platform roles.
- Strong hands-on experience with AWS services, and proficiency in Apache Iceberg or similar open table formats.
- Experience in a Government Commercial Cloud (GCC) environment is strongly preferred.
- AWS certifications such as AWS Certified Data Analytics or AWS Certified Solutions Architect are an advantage. Nice to Have:
- Hands-on experience with Microsoft Purview and Unity Catalog.
- Professional certifications in cloud, data engineering, data governance, or analytics.