Senior Big Data Backend Engineer - Remote (PySpark)

Binance; City Of Kalgoorlie-Boulder; Remote; AUD 120.000 - 190.000; Vollzeit

Benefits dieser Stelle

Learning budget

Performance bonuses

Wellbeing support

Zusammenfassung

You should bring strong Python/SQL/PySpark skills, Databricks cluster know-how, and experience delivering production-grade data pipelines in a SaaS setting. This is a remote-first role with opportunities to visit clients.

Qualifikationen

• Strong hands-on experience with Azure Databricks, Azure Data Factory, Azure SQL, Azure Storage, and Azure DevOps.
• Proficient in Python, SQL, and PySpark for large-scale distributed data processing.
• Deep experience configuring and managing Databricks clusters for scalable big data workloads.
• Experience building production-ready data pipelines in a SaaS or product-led engineering environment.
• Strong understanding of cloud-native data architecture, including data lakes, lakehouse, and modular pipeline design.

Aufgaben

• Design, develop, and maintain reusable in-house PySpark frameworks to enforce standardised data engineering patterns across the SaaS platform.
• Architect and implement scalable, production-grade ETL/ELT pipelines across Azure and AWS environments.
• Build distributed data processing solutions using Python and PySpark on Databricks.
• Develop batch and near real-time ingestion pipelines integrating third-party clinical systems, healthcare APIs, and external enterprise platforms.
• Design secure data integration patterns (REST APIs, SFTP, event-driven ingestion, webhooks) ensuring compliance and data integrity.
• Provide technical mentorship and enforce engineering standards across the analytics and data engineering team.

Kenntnisse

Python

SQL

PySpark

Azure Databricks

Data pipelines

Cloud architecture

CI/CD

Ausbildung

Bachelor's degree in CS/Data Science/Engineering

Tools

Azure Databricks

Azure Data Factory

Azure DevOps

Azure SQL

Azure Storage

AWS (S3, Lambda, RDS, Glue)

Databricks

Git

Jobbeschreibung

You should bring strong Python/SQL/PySpark skills, Databricks cluster know-how, and experience delivering production-grade data pipelines in a SaaS setting. This is a remote-first role with opportunities to visit clients.