Databricks Architect – Melbourne

Pan Asia Group; Melbourne; Hybrid; AUD 180,000 - 280,000; Full time

Job summary

Pan Asia Group is seeking a Databricks Architect in Melbourne to design and implement large-scale data solutions across cloud environments. You will lead lakehouse design, governance, and data migration initiatives for data engineering, data science and BI teams.

Required is deep Databricks experience, expert Spark, Python and SQL skills, and a proven record of optimal performance and cost control. Databricks or cloud architecture certifications are highly desirable.

Qualifications

• Bachelor's degree in Computer Science or a related technical field.
• 6+ years of experience in data warehousing, data engineering, or cloud architecture.
• 3+ years of specific, deep experience designing and deploying solutions on the Databricks platform.
• Expert proficiency in Spark, Python, and SQL.
• Certification as a Databricks Certified Data Engineer or Cloud Architect is highly desirable.

Responsibilities

• Lakehouse Design: Design and build optimized data architectures (e.g., Bronze, Silver, Gold layers) within the Databricks Lakehouse using Delta Lake.
• Platform Governance: Establish governance, security, and best practices for the Databricks environment, including cluster management, workspace security, and Unity Catalog implementation.
• Performance Tuning: Optimize Spark jobs and notebooks for maximum performance and cost-efficiency on Databricks clusters.
• Data Migration: Lead the technical planning and execution of migrating existing data warehouses and data lakes onto the Databricks platform.
• Integration: Architect integrations between Databricks and other enterprise tools, cloud services, and BI platforms (e.g., Tableau, Power BI).

Skills

Databricks

Spark

Python

SQL

Education

Bachelor's degree in Computer Science or related field

Tools

Databricks

Job description

The Databricks Architect is a subject matter expert in the Databricks Lakehouse Platform. This role involves designing and implementing large-scale data solutions, migration strategies, and best practices for data engineering, data science, and BI teams leveraging Databricks across a cloud environment.

Key Responsibilities

• Lakehouse Design: Design and build optimized data architectures (e.g., Bronze, Silver, Gold layers) within the Databricks Lakehouse using Delta Lake.
• Platform Governance: Establish governance, security, and best practices for the Databricks environment, including cluster management, workspace security, and Unity Catalog implementation.
• Performance Tuning: Optimize Spark jobs and notebooks for maximum performance and cost-efficiency on Databricks clusters.
• Data Migration: Lead the technical planning and execution of migrating existing data warehouses and data lakes onto the Databricks platform.
• Integration: Architect integrations between Databricks and other enterprise tools, cloud services, and BI platforms (e.g., Tableau, Power BI).

Required Qualifications

• Bachelor's degree in Computer Science or a related technical field.
• 6+ years of experience in data warehousing, data engineering, or cloud architecture.
• 3+ years of specific, deep experience designing and deploying solutions on the Databricks platform.
• Expert proficiency in Spark, Python, and SQL.
• Certification as a Databricks Certified Data Engineer or Cloud Architect is highly desirable.