Lead Data Engineer – Azure Pipelines & Hybrid Work

Crime and Corruption Commission QLD; City of Brisbane; Hybrid; AUD 110,000 - 150,000; Full time

Benefits offered by this job

Salary packaging

Superannuation 12.75%

Flexible work

Parental leave

Time off

Leave purchase

Annual leave loading

Wellbeing program

Study assistance

Training opportunities

Flu vaccination

City transport access

Job summary

The Crime and Corruption Commission QLD in Brisbane currently seeks a data engineer to join a hybrid team. You will design and implement data pipelines, integrate data from various sources, and maintain the Azure data estate while ensuring data quality and security.

Collaboration with data analysts and stakeholders is essential. Applicants should have 2+ years in data engineering and hands-on experience with Azure Data Factory, Azure SQL, Azure Data Lake, and Azure App Service.

Qualifications

• Two years of demonstrated experience in data engineering.
• Experience with Azure Data Factory, Azure SQL, Azure Data Lake and Azure App Service.
• Build and maintain data pipelines for data integration and processing.
• Ensure data quality and accuracy through testing and validations.
• Work with structured, unstructured and semi-structured data.
• Understand Kimball and Inmon data warehouse modelling.
• Monitor and maintain Azure data warehouse environments and optimize for security, performance and cost.
• Support machine learning workflows by building scalable data pipelines and collaborating with analysts.

Responsibilities

• Build and maintain data pipelines for data integration and processing.
• Ensure data quality and accuracy through testing and validations.
• Work with structured, unstructured, and semi-structured data.
• Monitor and maintain Azure data warehouse environments and optimize performance and cost.
• Support ML workflows by building pipelines and enabling feature engineering.

Skills

Data engineering

Azure Data Factory

Azure SQL

Azure Data Lake

Azure App Service

Data pipelines

Data quality

Kimball modelling

Inmon modelling

Machine learning

CI/CD

DevOps

BI

Education

Tertiary qualifications in data engineering / IT

Tools

Azure Data Factory

Azure SQL

Azure Data Lake

Azure App Service

Job description

The Crime and Corruption Commission QLD in Brisbane currently seeks a data engineer to join a hybrid team. You will design and implement data pipelines, integrate data from various sources, and maintain the Azure data estate while ensuring data quality and security.

Collaboration with data analysts and stakeholders is essential. Applicants should have 2+ years in data engineering and hands-on experience with Azure Data Factory, Azure SQL, Azure Data Lake, and Azure App Service.