Azure Cloud Data Engineer: Spark, Databricks & Lakehouse

Softtest Pays Pty; Darwin; On-site; AUD 110,000 - 160,000; Full time

Job summary

Softtest Pays Pty is seeking a data engineer to design and implement ETL/ELT processes for large data movements in cloud environments. You will optimize data for query performance in engines like Spark and Azure SQL, and collaborate with engineering, storage and analytics teams.

Experience with Azure Data Factory and Databricks, as well as data lake/lakehouse concepts, is highly valued. Security clearances and Australian eligibility are required.

Qualifications

• Experience developing ETL/ELT processes for large data movement in cloud environments.
• Experience preparing data for query performance in cloud compute engines (Spark, Azure SQL).
• Experience with engineering, storage and analytics services in cloud infrastructure.
• Experience with Azure Data Factory and Databricks.
• Experience/Knowledge of Data Lake and Lakehouses.

Skills

ETL/ELT development

Cloud data performance

Tools

Azure Data Factory

Databricks

Spark

Graph Databases

Azure SQL

Job description

Softtest Pays Pty is seeking a data engineer to design and implement ETL/ELT processes for large data movements in cloud environments. You will optimize data for query performance in engines like Spark and Azure SQL, and collaborate with engineering, storage and analytics teams.

Experience with Azure Data Factory and Databricks, as well as data lake/lakehouse concepts, is highly valued. Security clearances and Australian eligibility are required.