Senior Data Engineer: AI-Driven Lakehouse & Cloud Pipelines

Mantel Group Pty Ltd; Australia; On-site; AUD 140,000 - 190,000; Full time

Job summary

Mantel Group Pty Ltd in Melbourne, Sydney, Brisbane, and Perth seeks a Senior Data Engineer to design and build scalable data solutions. You will implement modern architectures (Lakehouse) on Databricks, develop ETL/ELT pipelines with Spark and Delta Lake, and ensure production readiness with CI/CD and monitoring.

The role requires deep SQL and Python (PySpark) or Scala, hands-on Databricks experience, and ability to work with cloud platforms across AWS, Azure, or GCP.

Qualifications

• Experience with Data engineering for production deployments.
• Design and implement data pipelines and architectures.
• Strong SQL and Python (PySpark) or Scala skills.
• Hands-on Databricks experience on AWS/GCP/Azure.
• Solid CI/CD, monitoring, and data quality practices.
• Ability to lead small teams and mentor others.
• Engage with senior stakeholders and clients.
• Experience with data governance and security considerations.

Responsibilities

• Work on complex data solutions meeting client expectations.
• Build and deploy modern data architectures on Databricks.
• Develop ETL/ELT pipelines with Spark and Delta Lake.
• Handle Big Data, PII, data wrangling and formatting.
• Apply best practices in CI/CD, monitoring, alerting, and IaC.
• Productionise AI models and data pipelines feeding live apps.

Skills

SQL

Python (PySpark)

Scala

Databricks

Spark processing

CI/CD

IaC

Data modeling

Data engineering

Stakeholder engagement

Tools

Databricks (Workflows)

Delta Live Tables

Unity Catalog

Cloud platforms (AWS/GCP/Azure)

Job description

Mantel Group Pty Ltd in Melbourne, Sydney, Brisbane, and Perth seeks a Senior Data Engineer to design and build scalable data solutions. You will implement modern architectures (Lakehouse) on Databricks, develop ETL/ELT pipelines with Spark and Delta Lake, and ensure production readiness with CI/CD and monitoring.

The role requires deep SQL and Python (PySpark) or Scala, hands-on Databricks experience, and ability to work with cloud platforms across AWS, Azure, or GCP.