Data Engineer — AWS, Spark & ETL Pipelines

Kaizen Global Technologies; Sydney; On-site; AUD 120,000 - 170,000; Full time

Job summary

Kaizen Global Technologies in Sydney, Australia, is seeking a data engineer to design, develop and maintain scalable ETL/ELT data pipelines. You will build data solutions using AWS services like S3, Glue, EMR, Athena and Aurora PostgreSQL, and develop data processing with Python, SQL, Spark and PySpark.

You will design data models, implement CDC and data integration processes, ensure quality and security, and collaborate with engineering, architecture, product and business teams in an Agile

Qualifications

• Experience building scalable ETL/ELT data pipelines.
• Strong AWS experience (S3, Glue, EMR, Athena, Aurora PostgreSQL).
• Proficient in Python and SQL for data processing.
• Experience with data governance, monitoring and CI/CD practices.
• Excellent problem-solving and stakeholder communication.

Responsibilities

• Design, develop and maintain scalable ETL/ELT data pipelines.
• Build and support data solutions using AWS S3, Glue, EMR, Athena and Aurora PostgreSQL.
• Develop data processing and transformation solutions using Python, SQL, Spark and PySpark.
• Design and maintain effective data models for analytical and operational requirements.
• Implement CDC and data integration processes.
• Ensure data quality, governance, security and reliability across data platforms.
• Troubleshoot production issues and optimise data pipelines for performance.
• Implement automated deployments and testing through CI/CD pipelines and GitHub Actions.
• Collaborate with engineering, architecture, product and business stakeholders in an Agile environment.

Skills

SQL

Python

AWS

Spark

ETL/ELT

Data modeling

CDC

CI/CD

Stakeholder mgmt

Tools

GitHub Actions

Job description

Kaizen Global Technologies in Sydney, Australia, is seeking a data engineer to design, develop and maintain scalable ETL/ELT data pipelines. You will build data solutions using AWS services like S3, Glue, EMR, Athena and Aurora PostgreSQL, and develop data processing with Python, SQL, Spark and PySpark.

You will design data models, implement CDC and data integration processes, ensure quality and security, and collaborate with engineering, architecture, product and business teams in an Agile