Senior Data Engineer: AWS Pipelines for ML & Analytics

Peoplebank; Sydney; On-site; AUD 140,000 - 190,000; Full time

Job summary

Peoplebank is seeking a Senior Data Engineer to design and deliver production data pipelines and data products for a government engagement. You will apply your expertise across AWS, Python, SQL and modern data engineering practices while collaborating with data scientists and ML engineers.

You will build scalable pipelines, implement data quality controls, metadata and lineage, and optimise cloud workloads.

Qualifications

• Production data pipelines and data products on AWS.
• ETL/ELT, data modelling and pipeline orchestration knowledge.
• Advanced Python and SQL, maintainable, efficient code.
• Experience with Git, automated testing, CI/CD and IaC.
• Cloud-based data solutions in containerised environments.
• Understanding of cloud security, access controls and monitoring.
• Strong communication and cross-team collaboration.
• Australian citizenship mandatory; NV1 clearance desirable.

Responsibilities

• Design, build, test and maintain scalable data pipelines on AWS.
• Create curated, reusable datasets and data products.
• Collaborate with data scientists and ML engineers on feature engineering and model pipelines.
• Translate business requirements into technical designs, estimates and acceptance criteria.
• Implement data quality, metadata, lineage, monitoring and alerting.
• Optimise Python, SQL and cloud workloads for performance and cost.
• Apply CI/CD, version control and infrastructure-as-code.
• Maintain data mappings, interface specs and operational runbooks.
• Coordinate with architecture, security, privacy, platform and delivery teams.

Skills

AWS

Python

SQL

ETL/ELT

Data modelling

Pipeline orchestration

Data quality testing

Metadata & lineage

Git

CI/CD

Infrastructure as code

Cloud security

Access controls

Secrets management

Monitoring

Communication

Collaboration

Tools

S3

Glue

Lambda

Athena

EMR

Kinesis

Step Functions

Terraform

AWS CloudFormation

Git

Job description

Peoplebank is seeking a Senior Data Engineer to design and deliver production data pipelines and data products for a government engagement. You will apply your expertise across AWS, Python, SQL and modern data engineering practices while collaborating with data scientists and ML engineers.

You will build scalable pipelines, implement data quality controls, metadata and lineage, and optimise cloud workloads.