Hybrid Data Engineer: Databricks & Azure

AC3; Sydney; Hybrid; AUD 203,000 - 221,000; Full time

Benefits offered by this job

Hybrid work arrangement

Competitive daily rate

Job summary

AC3 is seeking a Data Engineer in Sydney to help transition elements into the Databricks Medallion architecture. You will create and maintain data pipelines, build scalable data platforms, and model data for analytics.

You will collaborate with a broader analytics team, implement data protection processes, and communicate findings to stakeholders. Hybrid work with 2 days/week in the Sydney CBD office, ASAP start.

Qualifications

• 5+ years of experience as a Data Engineer.
• Experience developing processes that support data transformation, data structures, and the manipulation, processing, and extraction of value from datasets.
• Experience in machine learning, data mining, mathematics and statistical modelling and analysis.
• In depth knowledge of data platform concepts, including data lakes, data warehousing, big data processing, real-time processing architecture (ADF, Databricks, Logic Apps, Functions, Data Lake Gen2).
• Scheduling, and monitoring of ETL/ELT workflows.
• Experience with SQL, Python, SQL Server, NoSQL and Oracle databases.
• Experience with data pipeline and workflow management tools, and Cloud data platforms.
• Bachelor’s Degree in Data Science or a similar related field.

Responsibilities

• Create and maintain a data pipeline and infrastructure that utilizes diverse storage technologies.
• Develop a scalable data platforms using APIs and quality frameworks for increasing data volume and complexity.
• Assist in the establishment of processes and standards for data mining, data modeling, and data protection.
• Create data models into Databricks and Medallion architecture.
• Create clear and effective data visualisations to communicate findings to stakeholders.
• Collaborate with the broader data and analytics team to enhance data models.
• Assist with building and optimising data pipelines and infrastructure.

Skills

Databricks

SQL

Python

ETL/ELT

Data pipelines

Data modeling

Data lakes

Education

Bachelor's Degree in Data Science

Tools

SQL Server

NoSQL

Oracle

Logic Apps

Azure Data Factory (ADF)

Data Lake Gen2

Job description

AC3 is seeking a Data Engineer in Sydney to help transition elements into the Databricks Medallion architecture. You will create and maintain data pipelines, build scalable data platforms, and model data for analytics.

You will collaborate with a broader analytics team, implement data protection processes, and communicate findings to stakeholders. Hybrid work with 2 days/week in the Sydney CBD office, ASAP start.