Senior Databricks Platform Engineer: Security & Governance

Delivery Centric; Sydney; On-site; AUD 160,000 - 190,000; Full time

Job summary

Delivery Centric in Sydney is seeking a Senior Databricks Platform Administrator to manage enterprise Databricks environments, ensuring platform security, reliability, performance, governance, and cost optimisation. The role will supportdata, analytics, ML, and AI workloads while working closely with data, cloud, and security teams.

The role collaborates with data engineers, data scientists, MLOps, cloud, and security teams to deliver scalable compute, governance, and on-call production support.

Qualifications

• 5+ years of experience administering data or analytics platforms in enterprise environments.
• Hands-on experience administering Databricks workspaces, Spark-based platforms and clusters.
• Strong knowledge of Apache Spark architecture, execution, and performance tuning.
• Proficiency with Python and Bash scripting; SQL understanding required.
• Experience with Unity Catalog, Delta Lake, and data governance concepts.
• Cloud platform knowledge (Azure, AWS, or GCP) and cost optimisation awareness.

Responsibilities

• Administer enterprise Databricks workspaces across multiple environments.
• Configure clusters, jobs, pools, policies, secrets, and init scripts.
• Enforce cluster security, stability, and cost controls.
• Manage compute, autoscaling, and various cluster types.
• Oversee Databricks Jobs, Workflows, and scheduling for batch/streaming workloads.
• Implement and manage Unity Catalog for governance and access control.
• Integrate with cloud storage and external data sources.
• Monitor health using logs, metrics, and alerts; troubleshoot issues.
• Support ML/AI workloads with Databricks Runtime and Model Registry.
• Lead CI/CD for notebooks and configurations using Git workflows.
• Collaborate with data engineers, data scientists, MLOps, cloud and security teams.

Skills

Databricks administration

Spark architecture

Security concepts

CI/CD pipelines

Python scripting

Bash scripting

Cloud platforms

Data governance

Unity Catalog

Delta Lake

SQL

Education

Bachelor’s degree in Computer Science, Engineering, Data Engineering, or a related discipline

Tools

Terraform

Git-based tooling

Databricks REST APIs

CLI tools

Job description

Delivery Centric in Sydney is seeking a Senior Databricks Platform Administrator to manage enterprise Databricks environments, ensuring platform security, reliability, performance, governance, and cost optimisation. The role will supportdata, analytics, ML, and AI workloads while working closely with data, cloud, and security teams.

The role collaborates with data engineers, data scientists, MLOps, cloud, and security teams to deliver scalable compute, governance, and on-call production support.