EL2 Data Pipeline Engineer

Peoplebank; City of Melbourne; On-site; AUD 150,000 - 190,000; Full time

Job summary

Peoplebank in Melbourne is seeking an experienced EL2 Data Pipeline Engineer to design, build, test and maintain secure, scalable data pipelines that support a major digital transformation program.

You will collaborate with analysts and stakeholders to deliver reliable data integration across platforms, implement security controls, automate processes and contribute to Agile delivery practices.

Qualifications

• 5+ years of experience as a Data Pipeline Engineer in a cloud-based environment.
• Experience designing, building, testing and deploying data pipelines including APIs, MFTs and file extract processes.
• Strong experience with AWS data storage, extraction, transformation and processing services.
• Hands-on experience with AWS Glue, AWS Kinesis and HashiCorp Terraform.
• Solid understanding of cloud architecture, security principles and access controls for data platforms.
• Experience with ETL tools such as AWS Glue, Informatica, DBT, Talend or similar.
• Advanced SQL skills.
• Experience in Agile delivery environments (Scrum, Kanban, Lean).
• Strong problem-solving, analytical and communication skills.
• Relevant tertiary qualifications in IT or related discipline, or equivalent certifications.

Responsibilities

• Design, develop and implement scalable and secure data pipelines.
• Build, test, deploy and support data integration solutions including APIs, MFT and file-based integrations.
• Develop and maintain ETL processes and data transformation workflows.
• Monitor and optimise data pipelines for performance, reliability and scalability.
• Troubleshoot and resolve data integration and pipeline issues.
• Implement appropriate security controls and access management practices.
• Develop automation scripts to streamline operational processes.
• Collaborate with stakeholders, analysts and technical teams to understand data requirements.
• Produce and maintain technical documentation.
• Contribute to Agile delivery practices and continuous improvement.

Skills

5+ years experience

Data pipeline design

SQL

Agile delivery

Security principles

Cloud architecture

Education

Bachelor's degree in IT or related discipline

Tools

AWS Glue

AWS Kinesis

HashiCorp Terraform

GoAnywhere MFT

Ctrl-M orchestration

GitHub

Bitbucket

Informatica

DBT

Talend

Job description

About the Role We are seeking an experienced EL2 Data Pipeline Engineer to design, build, test and maintain secure, scalable data pipelines that support a major digital transformation program.

Working within a multidisciplinary project team, you will collaborate with technical specialists, analysts and stakeholders to deliver reliable data integration solutions that enable efficient data movement, processing and reporting across multiple platforms and systems.

Key Responsibilities

• Design, develop and implement scalable and secure data pipelines.
• Build, test, deploy and support data integration solutions, including APIs, managed file transfers (MFT) and file-based integrations.
• Develop and maintain ETL processes and data transformation workflows.
• Monitor and optimise data pipelines to ensure performance, reliability and scalability.
• Troubleshoot and resolve data integration and pipeline issues.
• Implement appropriate security controls and access management practices.
• Develop automation scripts to streamline operational processes.
• Collaborate with business stakeholders, analysts and technical teams to understand and deliver data requirements.
• Produce and maintain technical documentation.
• Contribute to Agile delivery practices and continuous improvement initiatives.

Essential Skills and Experience

• Minimum 5 years' experience as a Data Pipeline Engineer within a cloud-based environment.
• Demonstrated experience designing, building, testing and deploying data pipeline solutions, including APIs, managed file transfers and file extract processes.
• Strong experience with Amazon Web Services (AWS), particularly data storage, extraction, transformation and processing services.
• Hands-on experience with AWS Glue, AWS Kinesis and HashiCorp Terraform.
• Strong understanding of cloud architecture, security principles and access controls relating to data platforms.
• Experience with ETL tools such as AWS Glue, Informatica, DBT, Talend or similar technologies.
• Advanced SQL skills.
• Experience working in Agile delivery environments, including Scrum, Kanban or Lean methodologies.
• Strong problem-solving, analytical and communication skills.
• Relevant tertiary qualifications in Information Technology or related discipline, or equivalent industry certifications.

Desirable Skills and Experience

• Experience with GoAnywhere MFT.
• Experience with Ctrl-M orchestration.
• Experience with CI/CD automation and source control tools such as GitHub and Bitbucket.
• Experience applying Generative AI tools and spec-driven development practices to improve engineering outcomes.
• Understanding of architectural patterns for the safe and effective integration of Generative AI capabilities into the software development lifecycle.
• Previous experience in a similar senior data engineering role.

Key Attributes

• Strong stakeholder engagement and communication skills.
• Ability to work effectively across technical and business teams.
• Proactive and solution-focused approach.
• Strong attention to detail and commitment to quality.
• Ability to work in fast-paced, Agile delivery environments.

Peoplebank and Leaders IT are committed to creating a diverse and inclusive workplace where everyone belongs. We welcome applications from people of all backgrounds, identities, and experiences. If you need adjustments to the recruitment process due to your circumstances, please let us know—we’re here to support you.