Hybrid Data Engineer - Databricks, Delta Lake & PII

Myndshift Group; Queensland; Hybrid; AUD 122,000 - 150,000; Full time

Benefits offered by this job

Hybrid work up to 2 days from home

Superannuation up to 15.25%

Leave loading 17.5%

Wellbeing programs and discounts

Salary packaging options

Professional development support

Job summary

NHVR is seeking a Data Engineer to join the Data & Technology teams undergoing growth and transformation. You will design and operate scalable data pipelines in Databricks, focusing on governance, security, and reliable data delivery for regulatory processes.

Ideal candidates have four+ years of data engineering experience with Databricks or similar cloud platforms, strong Python/PySpark, and experience with Delta Lake and Unity Catalog. Hybrid work options are available.

Qualifications

• At least four years of data engineering experience with Databricks or equivalent cloud platform.
• Strong Python and PySpark with advanced SQL (Spark SQL & T-SQL or similar).
• Delta Lake experience including MERGE/upserts, schema evolution, OPTIMIZE/VACUUM.
• Experience with metadata-driven or configuration-driven pipelines.
• Solid Unity Catalog security and governance knowledge.
• Experience handling PII data with masking and least-privilege access.

Responsibilities

• Collaborate in a Scrum team to deliver BAU work and roadmap milestones.
• Interpret requirements, estimate workloads, and manage time for quality solutions.
• Design, build, and maintain scalable data pipelines in Databricks.
• Develop and operate Databricks notebooks and jobs; optimize performance and cost.
• Improve framework configurations for onboarding new data sources.
• Contribute to Lakeflow declarative pipelines and modern Databricks capabilities.
• Deliver changes via CI/CD, automated testing, and peer reviews.
• Support, maintain, and upgrade deployment pipelines and environments.
• Implement and manage Unity Catalog security, including permissions and secrets.
• Identify and protect sensitive data; apply tagging and masking.

Skills

Databricks

Python

PySpark

Spark SQL

SQL

Agile Scrum

CI/CD

Data modeling

Data governance

Security governance

Education

Bachelor's degree in a relevant field

Tools

Databricks notebooks

Lakeflow

Power BI

Tableau

Git/CI tools

Azure ADLS Gen2

Job description

NHVR is seeking a Data Engineer to join the Data & Technology teams undergoing growth and transformation. You will design and operate scalable data pipelines in Databricks, focusing on governance, security, and reliable data delivery for regulatory processes.

Ideal candidates have four+ years of data engineering experience with Databricks or similar cloud platforms, strong Python/PySpark, and experience with Delta Lake and Unity Catalog. Hybrid work options are available.