Senior AWS Data Pipeline Engineer - Secure & Scalable

Peoplebank; City of Melbourne; On-site; AUD 150,000 - 190,000; Full time

Job summary

Peoplebank in Melbourne is seeking an experienced EL2 Data Pipeline Engineer to design, build, test and maintain secure, scalable data pipelines that support a major digital transformation program.

You will collaborate with analysts and stakeholders to deliver reliable data integration across platforms, implement security controls, automate processes and contribute to Agile delivery practices.

Qualifications

• 5+ years of experience as a Data Pipeline Engineer in a cloud-based environment.
• Experience designing, building, testing and deploying data pipelines including APIs, MFTs and file extract processes.
• Strong experience with AWS data storage, extraction, transformation and processing services.
• Hands-on experience with AWS Glue, AWS Kinesis and HashiCorp Terraform.
• Solid understanding of cloud architecture, security principles and access controls for data platforms.
• Experience with ETL tools such as AWS Glue, Informatica, DBT, Talend or similar.
• Advanced SQL skills.
• Experience in Agile delivery environments (Scrum, Kanban, Lean).
• Strong problem-solving, analytical and communication skills.
• Relevant tertiary qualifications in IT or related discipline, or equivalent certifications.

Responsibilities

• Design, develop and implement scalable and secure data pipelines.
• Build, test, deploy and support data integration solutions including APIs, MFT and file-based integrations.
• Develop and maintain ETL processes and data transformation workflows.
• Monitor and optimise data pipelines for performance, reliability and scalability.
• Troubleshoot and resolve data integration and pipeline issues.
• Implement appropriate security controls and access management practices.
• Develop automation scripts to streamline operational processes.
• Collaborate with stakeholders, analysts and technical teams to understand data requirements.
• Produce and maintain technical documentation.
• Contribute to Agile delivery practices and continuous improvement.

Skills

5+ years experience

Data pipeline design

SQL

Agile delivery

Security principles

Cloud architecture

Education

Bachelor's degree in IT or related discipline

Tools

AWS Glue

AWS Kinesis

HashiCorp Terraform

GoAnywhere MFT

Ctrl-M orchestration

GitHub

Bitbucket

Informatica

DBT

Talend

Job description

Peoplebank in Melbourne is seeking an experienced EL2 Data Pipeline Engineer to design, build, test and maintain secure, scalable data pipelines that support a major digital transformation program.

You will collaborate with analysts and stakeholders to deliver reliable data integration across platforms, implement security controls, automate processes and contribute to Agile delivery practices.