Senior Data Engineer

Peoplebank; Sydney; On-site; AUD 140,000 - 190,000; Full time

Job summary

Peoplebank is seeking a Senior Data Engineer to design and deliver production data pipelines and data products for a government engagement. You will apply your expertise across AWS, Python, SQL and modern data engineering practices while collaborating with data scientists and ML engineers.

You will build scalable pipelines, implement data quality controls, metadata and lineage, and optimise cloud workloads.

Qualifications

• Production data pipelines and data products on AWS.
• ETL/ELT, data modelling and pipeline orchestration knowledge.
• Advanced Python and SQL, maintainable, efficient code.
• Experience with Git, automated testing, CI/CD and IaC.
• Cloud-based data solutions in containerised environments.
• Understanding of cloud security, access controls and monitoring.
• Strong communication and cross-team collaboration.
• Australian citizenship mandatory; NV1 clearance desirable.

Responsibilities

• Design, build, test and maintain scalable data pipelines on AWS.
• Create curated, reusable datasets and data products.
• Collaborate with data scientists and ML engineers on feature engineering and model pipelines.
• Translate business requirements into technical designs, estimates and acceptance criteria.
• Implement data quality, metadata, lineage, monitoring and alerting.
• Optimise Python, SQL and cloud workloads for performance and cost.
• Apply CI/CD, version control and infrastructure-as-code.
• Maintain data mappings, interface specs and operational runbooks.
• Coordinate with architecture, security, privacy, platform and delivery teams.

Skills

AWS

Python

SQL

ETL/ELT

Data modelling

Pipeline orchestration

Data quality testing

Metadata & lineage

Git

CI/CD

Infrastructure as code

Cloud security

Access controls

Secrets management

Monitoring

Communication

Collaboration

Tools

S3

Glue

Lambda

Athena

EMR

Kinesis

Step Functions

Terraform

AWS CloudFormation

Git

Job description

We’re seeking an experienced Senior Data Engineer to join a key government engagement, designing and delivering production-ready data pipelines and data products that support analytics, machine learning and AI initiatives.
This is an opportunity to apply your expertise across AWS, Python, SQL and modern data engineering practices while working closely with data scientists, machine learning engineers and multidisciplinary technology teams.

Key Responsibilities

• Design, build, test and maintain scalable data pipelines using AWS services.
• Develop curated, reusable and well-documented datasets and data products.
• Collaborate with data scientists and ML engineers to deliver data preparation, feature engineering and model input-output pipelines.
• Translate business requirements into technical designs, delivery estimates and acceptance criteria.
• Implement data quality controls, metadata, lineage, monitoring and alerting.
• Optimise Python, SQL and cloud workloads for performance, reliability and cost efficiency.
• Apply DevOps practices, including CI/CD, automated testing, version control and infrastructure-as-code.
• Maintain technical documentation, data mappings, interface specifications and operational runbooks.
• Work closely with architecture, cyber security, privacy, platform and delivery teams to ensure governance and operational requirements are met.

We’re seeking an experienced Senior Data Engineer to join a key government engagement, designing and delivering production-ready data pipelines and data products that support analytics, machine learning and AI initiatives.
This is an opportunity to apply your expertise across AWS, Python, SQL and modern data engineering practices while working closely with data scientists, machine learning engineers and multidisciplinary technology teams.

Key Responsibilities

• Design, build, test and maintain scalable data pipelines using AWS services.
• Develop curated, reusable and well-documented datasets and data products.
• Collaborate with data scientists and ML engineers to deliver data preparation, feature engineering and model input-output pipelines.
• Translate business requirements into technical designs, delivery estimates and acceptance criteria.
• Implement data quality controls, metadata, lineage, monitoring and alerting.
• Optimise Python, SQL and cloud workloads for performance, reliability and cost efficiency.
• Apply DevOps practices, including CI/CD, automated testing, version control and infrastructure-as-code.
• Maintain technical documentation, data mappings, interface specifications and operational runbooks.
• Work closely with architecture, cyber security, privacy, platform and delivery teams to ensure governance and operational requirements are met.

You will bring:

• Demonstrated experience building production data pipelines and data products on AWS, using services such as S3, Glue, Lambda, Athena, EMR, Kinesis or Step Functions.
• Strong knowledge of ETL/ELT, data modelling, pipeline orchestration, data quality testing, metadata and lineage.
• Advanced Python and SQL skills, including writing maintainable, testable code and optimising complex queries.
• Experience with Git, code reviews, automated testing, CI/CD and infrastructure-as-code using Terraform or AWS CloudFormation.
• Experience delivering cloud-based data solutions in containerised environments.
• A sound understanding of cloud security, access controls, secrets management and operational monitoring.
• Strong communication skills and the ability to collaborate across technical and business teams.

Australian citizenship is mandatory. An existing NV1 clearance is desirable.

Peoplebank and Leaders IT are committed to creating a diverse and inclusive workplace where everyone belongs. We welcome applications from people of all backgrounds, identities, and experiences. If you need adjustments to the recruitment process due to your circumstances, please let us know—we’re here to support you.

Be careful - Don’t provide your bank or credit card details when applying for jobs. Don't transfer any money or complete suspicious online surveys. If you see something suspicious, report this job ad.