Remote/Hybrid Data Engineer for AI-Driven Pipelines

Datasmiths; Australia; Hybrid; AUD 110,000 - 150,000; Full time

Job summary

Datasmiths is seeking a Data Engineer in Australia with 3+ years of experience to design and optimise robust data architectures. You will build scalable ETL pipelines, data lakes, and data platforms using Databricks, Spark, and Delta Lake across AWS/Azure.

You’ll collaborate with data scientists, analysts, and engineers to deliver AI-powered insights while enhancing data quality, governance, and automation across cloud-native solutions.

Qualifications

• 3+ years of experience in data engineering, big data, or cloud data solutions.
• Proficiency in Python, Scala, or SQL for building scalable data pipelines.
• Experience with Databricks, Apache Spark, and Delta Lake.
• Familiarity with cloud platforms (AWS, Azure) and modern data architectures.
• Understanding of ETL processes, data lakes, and warehouse solutions.

Responsibilities

• Build scalable data architectures and high‑performance data pipelines.
• Optimise big data processing with cloud-native solutions.
• Ensure data quality, governance, and security across data platforms.
• Drive automation of data workflows using Python, Scala, or SQL.
• Collaborate with data scientists, analysts, and engineers to deliver data products.
• Stay updated on data engineering trends, AI advancements, and cloud technologies.

Skills

Python

Scala

SQL

Databricks

Apache Spark

Delta Lake

AWS

Azure

ETL

Data Lakes

Tools

Databricks

Job description

Datasmiths is seeking a Data Engineer in Australia with 3+ years of experience to design and optimise robust data architectures. You will build scalable ETL pipelines, data lakes, and data platforms using Databricks, Spark, and Delta Lake across AWS/Azure.

You’ll collaborate with data scientists, analysts, and engineers to deliver AI-powered insights while enhancing data quality, governance, and automation across cloud-native solutions.