Cloud Data Engineer – ETL/ELT on Azure Databricks

TaskVerse; Australia; On-site; AUD 120,000 - 180,000; Full time

Job summary

Softtest Pays Pty is seeking an experienced Data Engineer to design, develop and optimize ETL/ELT processes for large-scale data workloads in a cloud environment. You will work across ingestion, transformation and query-ready outputs to ensure data is structured for cloud analytics.

Applicants should have practical cloud data platform experience and familiarity with Spark, Azure Data Factory, Databricks and data lakes or lakehouses.

Qualifications

• Experience developing ETL/ELT for complex or large-scale data workloads.
• Strong ability to prepare data for query performance in cloud compute engines.
• Familiarity with SQL and data platform concepts; Azure Data Factory and Databricks knowledge is preferred.

Responsibilities

• Design, develop, test and maintain ETL/ELT workflows across cloud platforms.
• Prepare and organize data to improve query performance in cloud environments.
• Collaborate with engineering, storage and analytics teams to deliver integrated data solutions.

Skills

ETL/ELT

Cloud data platforms

SQL

Spark

Education

Bachelor’s degree in CS or related

Tools

Azure Data Factory

Databricks

Azure SQL

Spark

Job description

Softtest Pays Pty is seeking an experienced Data Engineer to design, develop and optimize ETL/ELT processes for large-scale data workloads in a cloud environment. You will work across ingestion, transformation and query-ready outputs to ensure data is structured for cloud analytics.

Applicants should have practical cloud data platform experience and familiarity with Spark, Azure Data Factory, Databricks and data lakes or lakehouses.