Machine Learning Engineer

Bailey Abbott Pty Ltd; Perth; On-site; AUD 110,000 - 170,000; Full time

Job summary

Bailey Abbott Pty Ltd in Australia is seeking a Machine Learning Engineer with a strong software engineering foundation to productionise, scale, and maintain advanced analytical models. You will bridge data science experimentation and robust software, designing scalable data pipelines within our Databricks environment using Spark and R.

You will collaborate with data architects, engineers, and analysts to integrate predictive outputs into downstream systems, ensure pipeline reliability,

Qualifications

• Experience building production-grade ML pipelines in Spark/Databricks.
• Strong software engineering practices including testing and CI/CD.
• Proficiency in R for production analytics and Spark integrations.
• Experience with Delta Lake and large-scale time-series data.
• Familiarity with Python/PySpark in cross-language workflows.

Responsibilities

• Design scalable data pipelines and automated ML lifecycles.
• Collaborate with data architects, engineers, and analysts to integrate predictive outputs into downstream systems.
• Monitor performance, pipeline latency, and ensure data governance across production workspaces.
• Bridge data science experiments with robust software production processes.
• Develop ML deployment and retraining workflows (CI/CD, MLflow).

Skills

Software Engineering

Databricks

Apache Spark

R for production analytics

Data querying & Delta Lake

Python / PySpark

CI/CD pipelines

Tools

MLflow

Unity Catalog

Sparklyr / SparkR

Terraform

Databricks Workspace

Job description

Our Company

We exist to bring amazing people together to explore the art of possible. Bailey Abbott are a progressive and dynamic IT Consultancy business working across both Public and Private sectors. People are at the core of everything we do. We're all about nurturing and inspiring people - ours and our clients. We bring a positive attitude and mindset to everything we do. We confidently use our knowledge and skills to solve problems, finding a better and easier way forward. We deliver transformational outcomes to take businesses further, fearlessly. Confidence and trust is fundamental to delivering superior outcomes and we take ownership and responsibility for our commitments.

Our People

Our staff are creative, collaborative, and passionate. We're strong advocates for change, challenging the status quo through new thinking, technology, and practices. Always exploring creative ways to simplify complexity.

Role Overview

We are seeking a Machine Learning Engineer with a strong software engineering foundation to productionise, scale, and maintain advanced analytical and predictive models. In this role, you will bridge the gap between data science experimentation and robust software engineering, designing scalable data pipelines and automated ML lifecycles within our Databricks environment using Spark, R, and modern engineering practices.

Key Responsibilities

• Production ML Pipelines: Build, test, and optimize production-grade data processing and model training pipelines in Apache Spark and Databricks.
• MLOps & Lifecycle Automation: Standardise model deployment, versioning, monitoring, and automated retraining workflows (CI/CD, MLflow).
• Code Quality & Architecture: Apply software engineering best practices-modular code architecture, automated testing, containerisation, and version control-to analytical and statistical codebases.
• R & Statistical Workloads: Optimise and scale R-based modeling workflows to run efficiently over distributed data architectures using Sparklyr/SparkR.
• Collaboration & Integration: Partner with data architects, data engineers, and domain analysts to integrate predictive outputs cleanly into downstream systems and analytical platforms.
• Performance & Reliability: Monitor pipeline latency, manage compute cluster sizing, and ensure data integrity and governance across production workspaces.

Required Skills & Experience

• Software Engineering Fundamentals: Solid foundation in core software engineering principles (design patterns, unit/integration testing, Git version control, CI/CD pipelines).
• Databricks Ecosystem: Proven hands-on experience orchestrating workloads, managing workspaces, and using MLflow / Unity Catalog within Databricks.
• Distributed Computing (Apache Spark): Deep practical understanding of Spark internals, data partitioning, query optimisation, and memory management.
• R for Production / Advanced Analytics: Strong proficiency in R (including Spark integrations like sparklyr or SparkR) for statistical modeling, forecast automation, or data manipulation.
• Data Layer Experience: Comfort querying complex relational data, working with Delta Lake formats, and handling large-scale time-series or tabular datasets.
• Desirable Skills: Familiarity with Python/PySpark alongside R for cross-language workflows. Experience with cloud infrastructure (Azure, AWS, or GCP) and Infrastructure as Code (Terraform). Understanding of model drift detection, data lineage, and enterprise data governance frameworks.

Additional Information

Our clients are diverse and so are we. We engage with great talent from all walks of life to bring their extensive and varied experience to help promote innovation.

We encourage applications from candidates from all backgrounds to further strengthen Bailey Abbott.

Bailey Abbott. Explore Possible.