Production ML Engineer: Databricks & MLOps

Bailey Abbott Pty Ltd; Perth; On-site; AUD 110,000 - 170,000; Full time

Job summary

Bailey Abbott Pty Ltd in Australia is seeking a Machine Learning Engineer with a strong software engineering foundation to productionise, scale, and maintain advanced analytical models. You will bridge data science experimentation and robust software, designing scalable data pipelines within our Databricks environment using Spark and R.

You will collaborate with data architects, engineers, and analysts to integrate predictive outputs into downstream systems, ensure pipeline reliability,

Qualifications

• Experience building production-grade ML pipelines in Spark/Databricks.
• Strong software engineering practices including testing and CI/CD.
• Proficiency in R for production analytics and Spark integrations.
• Experience with Delta Lake and large-scale time-series data.
• Familiarity with Python/PySpark in cross-language workflows.

Responsibilities

• Design scalable data pipelines and automated ML lifecycles.
• Collaborate with data architects, engineers, and analysts to integrate predictive outputs into downstream systems.
• Monitor performance, pipeline latency, and ensure data governance across production workspaces.
• Bridge data science experiments with robust software production processes.
• Develop ML deployment and retraining workflows (CI/CD, MLflow).

Skills

Software Engineering

Databricks

Apache Spark

R for production analytics

Data querying & Delta Lake

Python / PySpark

CI/CD pipelines

Tools

MLflow

Unity Catalog

Sparklyr / SparkR

Terraform

Databricks Workspace

Job description

Bailey Abbott Pty Ltd in Australia is seeking a Machine Learning Engineer with a strong software engineering foundation to productionise, scale, and maintain advanced analytical models. You will bridge data science experimentation and robust software, designing scalable data pipelines within our Databricks environment using Spark and R.

You will collaborate with data architects, engineers, and analysts to integrate predictive outputs into downstream systems, ensure pipeline reliability,