Senior Data Platform Engineer - Databricks & AI Pipelines

Nuix; Sydney; Hybrid; AUD 180,000 - 240,000; Full time

Job summary

Nuix is seeking a Senior Data Platform Engineer to design and oversee Databricks pipelines on AWS, delivering scalable, governance-driven data for analytics and AI workloads. You’ll transform raw data from multiple systems into analytics-ready datasets and collaborate across a cross-functional Data, Analytics & AI team.

You will contribute to pipeline engineering, data transformation, and governance, while occasionally engaging with platform engineering and dashboard tasks to support broader

Qualifications

• Experience building data ingestion pipelines (Databricks Auto Loader / Lakeflow Connectors / Fivetran).
• Strong SQL, Python and Spark (PySpark) for large-scale processing.
• Experience with Databricks Workflows, Jobs, DLT or Lakeflow pipelines.
• Deep understanding of CDC, SCD and incremental data loading in a Lakehouse.
• Design of Medallion Architecture (bronze/silver/gold) using Delta Lake.
• Data quality checks and observability with frameworks like Great Expectations or DL T.
• Data cataloguing, lineage, and metadata management in Unity Catalog.
• Spark tuning and Delta Lake optimization; production-grade pipelines.

Responsibilities

• Design, build and maintain scalable ETL/ELT pipelines for analytics and AI workloads.
• Build data integrations with SaaS platforms like Salesforce, NetSuite, Jira, etc.
• Implement incremental and historical data processing for up-to-date datasets.
• Ensure data quality, reliability and performance across pipelines.
• Support data governance, lineage, and access controls.
• Operate production pipelines with monitoring and incident response.
• Collaborate with stakeholders to translate requirements into datasets.
• Contribute to team standards, docs and engineering best practices.

Skills

SQL

Python

Apache Spark (PySpark)

Databricks

Data Ingestion

Data Transformation

Data Governance

DevOps

CI/CD

Stakeholder Communication

Production Engineering

Tools

Databricks

AWS

Delta Lake

Unity Catalog

Fivetran

Lakeflow

Git

Job description

Nuix is seeking a Senior Data Platform Engineer to design and oversee Databricks pipelines on AWS, delivering scalable, governance-driven data for analytics and AI workloads. You’ll transform raw data from multiple systems into analytics-ready datasets and collaborate across a cross-functional Data, Analytics & AI team.

You will contribute to pipeline engineering, data transformation, and governance, while occasionally engaging with platform engineering and dashboard tasks to support broader