Azure data engineer

Spait Infotech; Victoria; Presencial; AUD 95.000 - 130.000; Jornada completa

Descripción de la vacante

Spait Infotech in Victoria, Australia, seeks a data engineer to design and manage modern data pipelines using Azure technologies. You will transform data with Databricks, write SQL, and collaborate with analysts to deliver trusted data assets.

The role emphasizes data quality, security, and scalable ETL/ELT processes, with hands-on experience in ADF, ADLS, PySpark, and Python.

Formación

• Hands-on experience with Azure cloud data services (ADF, ADLS, Databricks).
• Strong SQL querying and database knowledge for data retrieval.
• Proficiency in Python for data processing and scripting.
• Understanding of ETL and ELT concepts and best practices.

Responsabilidades

• Build and optimize data pipelines using Azure Databricks and PySpark.
• Write and tune SQL queries for data extraction and transformation.
• Work with Azure Data Lake Storage to store and manage data assets.
• Collaborate with business analysts and development teams to deliver reliable data.
• Ensure data quality, security, and pipeline reliability; monitor and troubleshoot.

Conocimientos

Azure Databricks

SQL

Python

PySpark

Azure Data Factory

Azure Data Lake Storage

ETL/ELT concepts

Git

CI/CD

Data Quality

Herramientas

ADF (Azure Data Factory)

Azure Data Lake Storage

Azure Databricks

PySpark

Descripción del empleo

• Work with Azure Databricks for data processing and transformation.
• Write and optimize SQL queries.
• Work with Azure Data Lake Storage for storing and managing data.
• Build ETL/ELT workflows for data integration.
• Perform data cleaning, transformation, and validation.
• Monitor and troubleshoot data pipelines.
• Work with business analysts and development teams.
• Ensure data quality, security, and reliability.

Required Technical Skills

• Good knowledge of Microsoft Azure.
• Hands-on experience with Azure Data Factory (ADF).
• Knowledge of Azure Databricks and PySpark.
• Strong SQL and database knowledge.
• Experience with Python programming.
• Understanding of ETL and ELT concepts.
• Knowledge of Azure Data Lake Storage (ADLS).
• Basic understanding of data warehousing concepts.
• Knowledge of Git and CI/CD is an added advantage.
• Good analytical and problem-solving skills.