Cloud Data Engineer — Fabric/Databricks & AI Pipelines

Blackbook.AI; City of Brisbane; Hybrid; AUD 100,000 - 160,000; Full time

Benefits offered by this job

Competitive Remuneration

Flexible Working Conditions

Gym and PT

Free Spotify Account

Job summary

Blackbook.ai is a leading Australian AI company delivering data-driven outcomes for clients. We are seeking a Data Engineer to design, build and operate modern cloud data platforms across Microsoft Fabric and Databricks, with pipelines, warehousing and reporting functions.

You will work directly with clients, gather requirements and implement scalable data transformations in Python, SQL and PySpark while applying best practices and collaborating with cross-functional teams.

Qualifications

• Bachelor's degree or higher.
• Proficiency in Python and SQL for data engineering.
• Hands-on experience with Microsoft Fabric (Lakehouse, Warehouse, Notebooks and Pipelines).
• Building and orchestrating pipelines in Azure Data Factory or Fabric Pipelines.
• Cloud data warehousing using medallion architecture (bronze/silver/gold).
• Designing and building star schema dimensional models.
• Commercial experience with Databricks and PySpark.
• Reporting and data visualization, ideally with Power BI.
• Appropriate use of AI tools in data engineering work.
• Strong presentation and communication skills in writing and speaking.
• Critical thinking and problem solving abilities.
• Experience with AWS data engineering or willingness to upskill.

Responsibilities

• Work directly with clients as a Blackbook.ai consultant.
• Gather requirements, design, build and test data and analytics solutions.
• Build and orchestrate data pipelines using Fabric Pipelines and Azure Data Factory.
• Develop scalable data transformations in Python, PySpark and SQL across Fabric and Databricks.
• Design and implement cloud data warehouses and lakehouses with medallion architecture.
• Build star schema dimensional models for scalable, cost-efficient reporting.
• Deliver reporting and semantic models that business users can trust.
• Design and build metadata-driven data ingestion into data lakes, lakehouses and databases.
• Use AI tools to accelerate delivery and extract value from AI.
• Apply engineering best practices: Git, code reviews, automated testing, CI/CD.
• Take ownership of end-to-end process from requirements to implementation.
• Keep up with emerging technologies and share knowledge with peers.

Skills

Python

SQL

Problem solving

Communication

Education

Bachelor's degree in a related field

Tools

Microsoft Fabric

Azure Data Factory

Databricks

PySpark

Power BI

Git

CI/CD

Job description

Blackbook.ai is a leading Australian AI company delivering data-driven outcomes for clients. We are seeking a Data Engineer to design, build and operate modern cloud data platforms across Microsoft Fabric and Databricks, with pipelines, warehousing and reporting functions.

You will work directly with clients, gather requirements and implement scalable data transformations in Python, SQL and PySpark while applying best practices and collaborating with cross-functional teams.