Senior Data Engineer: Snowflake, PySpark & Data Pipelines

HCLTech - Australia and New Zealand; Sydney; On-site; AUD 120,000 - 180,000; Full time

Job summary

HCLTech in Australia and New Zealand is seeking a Data Engineer in Sydney to design and operate data pipelines using Snowflake, PySpark, Python and SQL. You will build scalable collections pipelines and work with cross-functional teams to ensure robust data workflows.

You will mentor Senior Data Engineers and Data Engineers, define data contracts and schema evolution, and establish production monitoring, CI/CD and data quality controls to support payments-related data initiatives.

Qualifications

• Experience in enterprise data architecture and systems integration.
• Proficient with Snowflake data warehousing and PySpark/Spark.
• Strong Python and advanced SQL skills, and knowledge of AWS.

Responsibilities

• Design the Snowflake and PySpark architecture for collections pipelines.
• Lead integration efforts and model data for collections use cases.
• Define data contracts and schema evolution standards.
• Establish automated data quality and reconciliation controls.
• Design restartable, idempotent and recoverable pipelines.
• Optimize Snowflake and PySpark workloads and monitor pipelines.
• Lead technical reviews and data investigations; mentor engineers.
• Collaborate with Cards MOD, PCC and platform teams.

Skills

Enterprise architecture and systems

Snowflake

PySpark / Spark

Python & SQL

AWS

Data contracts & schema evolution

APIs & distributed systems

CI/CD & production engineering

Job description

HCLTech in Australia and New Zealand is seeking a Data Engineer in Sydney to design and operate data pipelines using Snowflake, PySpark, Python and SQL. You will build scalable collections pipelines and work with cross-functional teams to ensure robust data workflows.

You will mentor Senior Data Engineers and Data Engineers, define data contracts and schema evolution, and establish production monitoring, CI/CD and data quality controls to support payments-related data initiatives.