Senior Big Data Engineer – Hybrid Sydney, Lead Pipelines

Paytech Nexus; Sydney; Hybrid; AUD 150,000 - 180,000; Full time

Job summary

Paytech Nexus in Sydney is seeking a Senior Data Engineer to lead data platform and pipeline initiatives across the bank’s big data and data warehouse systems. You will design scalable ingestion, transformation, and loading pipelines using Spark, Hadoop, Hive, and Kafka, while guiding a team of engineers and collaborating with internal stakeholders.

You're expected to drive best practices, implement security controls, and provide technology roadmaps for a multi-year data transformation journey,

Qualifications

• 5+ years hands-on data engineering experience in big data and data warehouse contexts.
• Experience with Hadoop-based data solutions, Spark, Kafka, and ETL frameworks.
• Strong Unix/Linux, SQL, and scripting in Scala/Java/Python.
• Experience leading engineers and collaborating with stakeholders.

Responsibilities

• Design, build and deliver enterprise data ingestion and pipeline solutions.
• Mentor data engineers and manage stakeholder relationships.
• Develop blueprints and roadmaps for multi-year data transformations.
• Ensure data quality and security controls across platforms.

Skills

Big Data

Scala

Java

Python

SQL

Data Lake

Data Warehousing

Tools

Hadoop (Cloudera)

Spark

MapReduce

Hive

Sqoop

Kafka

Teradata

AWS

Ab Initio

Cloudera CDP

Job description

Paytech Nexus in Sydney is seeking a Senior Data Engineer to lead data platform and pipeline initiatives across the bank’s big data and data warehouse systems. You will design scalable ingestion, transformation, and loading pipelines using Spark, Hadoop, Hive, and Kafka, while guiding a team of engineers and collaborating with internal stakeholders.

You're expected to drive best practices, implement security controls, and provide technology roadmaps for a multi-year data transformation journey,