Senior Azure Data Engineer - Databricks & Lakehouse

XPT Software; Canberra; Presencial; AUD 150.000 - 190.000; Jornada completa

Descripción de la vacante

XPT Software in Canberra, ACT, is seeking a Senior Data Engineer with strong hands-on experience in Azure Databricks to design, build, and optimize scalable data pipelines and lakehouse solutions for analytics and decision-making.

You will implement batch and streaming pipelines, integrate with Azure Data Lake Storage Gen2, Data Factory, and Key Vault, and deliver production-grade notebooks, jobs, and CI/CD workflows while enforcing data governance and cost efficiency.

Formación

• 8+ years of experience in data engineering or data platform delivery.
• Hands-on Azure Databricks expertise with notebooks, jobs, and workflows.
• Advanced Python, PySpark, and SQL proficiency for production pipelines.
• Experience with Delta Lake and lakehouse architectures.
• Experience with Azure Data Lake Storage Gen2 and Azure Data Factory.
• CI/CD in Azure DevOps and version control with Git.
• Knowledge of dimensional modeling, metadata, and data governance.
• Strong communication and collaboration skills.

Responsabilidades

• Design, build, and optimize scalable batch and streaming data pipelines on Azure Databricks.
• Develop reusable ingestion, transformation, validation, and publishing frameworks.
• Implement Delta Lake features and ACID transactions.
• Create production-grade notebooks, jobs, and cluster configurations.
• Monitor pipelines, troubleshoot issues, and perform root-cause analysis.
• Partner with Architects, Analysts, and stakeholders to translate requirements into data products.
• Lead code reviews, establish engineering standards, and mentor peers.
• Build CI/CD pipelines for code promotion and deployment across environments.

Conocimientos

Data engineering experience

Cloud data architecture

Strong communication

Problem solving

Code reviews

Stakeholder collaboration

Herramientas

Azure Databricks

Azure Data Lake Storage Gen2

Azure Data Factory

Delta Lake

Azure DevOps

PySpark

Python

SQL

Descripción del empleo

XPT Software in Canberra, ACT, is seeking a Senior Data Engineer with strong hands-on experience in Azure Databricks to design, build, and optimize scalable data pipelines and lakehouse solutions for analytics and decision-making.

You will implement batch and streaming pipelines, integrate with Azure Data Lake Storage Gen2, Data Factory, and Key Vault, and deliver production-grade notebooks, jobs, and CI/CD workflows while enforcing data governance and cost efficiency.