Senior Data Backend Engineer (PySpark/Databricks) Remote

Binance; City of Greater Shepparton; Remote; AUD 120,000 - 160,000; Full time

Benefits offered by this job

Learning budget

Performance bonuses

Wellbeing support

Remote-first organisation

Job summary

Halo Labs is seeking a hands-on Data Engineer to design, develop, and maintain PySpark frameworks for our SaaS platform. You will architect scalable ETL/ELT pipelines across Azure and AWS and build distributed data processing with Python and PySpark on Databricks.

You will collaborate with software engineers to embed data services, optimise Spark jobs and data models, and implement CI/CD, IaC, and data governance to ensure reliability, security, and performance.

Qualifications

• Proven experience with Python, SQL, and PySpark for large-scale distributed data processing.
• Deep knowledge of cloud-native data architectures including lakehouse and data lakes.
• Experience building production-ready data pipelines in a SaaS or product-led environment.

Responsibilities

• Design, develop, and maintain reusable in-house PySpark frameworks for standardised data engineering patterns.
• Architect scalable ETL/ELT pipelines across Azure and AWS environments.
• Build distributed data processing solutions using Python and PySpark on Databricks.

Skills

Python

SQL

PySpark

Azure

AWS

Databricks

Data pipelines

SaaS

CI/CD

GIT/AZURE DEVOPS

Education

Degree in Computer Science/Engineering/Data Science

Tools

Databricks

Azure DevOps

Azure Data Factory

Azure SQL

AWS services (S3/Lambda/RDS/Glue)

Job description

Halo Labs is seeking a hands-on Data Engineer to design, develop, and maintain PySpark frameworks for our SaaS platform. You will architect scalable ETL/ELT pipelines across Azure and AWS and build distributed data processing with Python and PySpark on Databricks.

You will collaborate with software engineers to embed data services, optimise Spark jobs and data models, and implement CI/CD, IaC, and data governance to ensure reliability, security, and performance.