Cloud Data Engineer – Azure, Spark & Data Lake

Softtest Pays Pty; Queensland; On-site; AUD 120,000 - 160,000; Full time

Job summary

Softtest Pays Pty is seeking experienced data engineering professionals in Australia who can design and implement scalable ETL/ELT pipelines in cloud environments. The role emphasizes data movement, transformation, and visualisation at scale, with a focus on Azure Data Factory, Databricks and lakehouse architectures.

Candidates should be proficient in Spark, SQL, and analytics, and capable of collaborating with Engineering, Storage and Analytics teams to optimise data systems in cloud

Qualifications

• Experience developing ETL/ELT processes for complex and/or large data movement, transformation or visualisations, especially in a cloud environment.
• Experience preparing data optimised for query performance in cloud compute engines (e.g., Spark), graph databases, and Azure SQL.
• Experience collaborating with Engineering, Storage and Analytics services in cloud infrastructure.

Responsibilities

• Develop ETL/ELT pipelines for large data movement and cloud-based transformations.
• Optimise data structures for fast query performance in distributed cloud engines.
• Collaborate with Engineering, Storage and Analytics teams in cloud infrastructure projects.

Skills

Spark

Storage

Analytics

Infrastructure

Databases

Security

SQL

Engineering

Tools

Azure Data Factory

Databricks

Data Lake

Lakehouses

Job description

Softtest Pays Pty is seeking experienced data engineering professionals in Australia who can design and implement scalable ETL/ELT pipelines in cloud environments. The role emphasizes data movement, transformation, and visualisation at scale, with a focus on Azure Data Factory, Databricks and lakehouse architectures.

Candidates should be proficient in Spark, SQL, and analytics, and capable of collaborating with Engineering, Storage and Analytics teams to optimise data systems in cloud