Senior Data Platform Engineer - API & Pipelines (Remote)

Crowdstrike Inc.; Melbourne; Remote; AUD 120,000 - 170,000; Full time

Benefits offered by this job

Dedicated learning budget

Performance bonuses

Wellbeing support

Remote-first organisation

Job summary

Halo Labs is a future-focused data solutions firm delivering secure, data-driven software for the SaaS landscape. We seek a Software Developer to design, build, and maintain scalable data services that power modern analytics platforms.

You’ll work on PySpark-based frameworks, Databricks environments, and cloud-native pipelines across Azure and AWS, collaborating with cross-functional teams to turn business requirements into robust data solutions.

Qualifications

• Strong hands-on experience with Azure services including Databricks, Azure Data Factory, Azure SQL, Azure Storage, and Azure DevOps.
• Practical experience with AWS services (S3, Lambda, RDS, Glue, IAM).
• Advanced proficiency in Python, SQL, and PySpark for large-scale distributed data processing.
• Experience building production-ready data pipelines in a SaaS or product-led engineering environment.
• Degree in Computer Science, Engineering, Data Science, or related field.

Responsibilities

• Design, develop, and maintain reusable PySpark frameworks to enforce standardized data engineering patterns across the SaaS platform.
• Architect and implement scalable, production-grade ETL/ELT pipelines across Azure and AWS environments.
• Build distributed data processing solutions using Python and PySpark on Databricks.
• Develop batch and near real-time ingestion pipelines integrating third-party clinical systems, healthcare APIs, and external enterprise platforms.
• Design secure data integration patterns (REST APIs, SFTP, event-driven ingestion, webhooks) ensuring compliance and data integrity.
• Work closely with Software Engineers to embed data services directly into the SaaS product architecture.
• Contribute to backend system design discussions to ensure data layer scalability, observability, and performance.
• Implement CI/CD pipelines using Azure DevOps, Git, and Azure Pipelines for automated deployment and testing of data workloads.
• Apply infrastructure-as-code and environment management best practices across Azure and AWS.
• Optimise Spark jobs, cluster configurations, and storage strategies for performance and cost efficiency.
• Design and maintain robust data models, including dimensional models and SaaS-oriented data schemas.
• Implement data validation, monitoring, and alerting to ensure pipeline reliability and production stability.
• Provide technical mentorship and enforce engineering standards across the analytics and data engineering team.

Skills

Python

SQL

PySpark

Databricks

Azure

AWS

Data pipelines

CI/CD

Education

Bachelor's degree in CS/Engineering/DS

Tools

Azure DevOps

Git

Azure Pipelines

Azure Data Factory

S3

Lambda

RDS

Glue

REST APIs

Job description

Halo Labs is a future-focused data solutions firm delivering secure, data-driven software for the SaaS landscape. We seek a Software Developer to design, build, and maintain scalable data services that power modern analytics platforms.

You’ll work on PySpark-based frameworks, Databricks environments, and cloud-native pipelines across Azure and AWS, collaborating with cross-functional teams to turn business requirements into robust data solutions.