Data Engineer

Silverlink Technologies; Brisbane; On-site; AUD 120,000 - 180,000; Full time

Job summary

Silverlink Technologies is seeking a Senior Data Engineer with strong Databricks and AWS expertise to design, develop, optimize, and support enterprise-grade data pipelines. The role emphasizes production BAU operations, data quality, and compliance with ITIL/ITSM practices.

You'll monitor pipelines, resolve incidents, and ensure SLAs across cloud-based data platforms. Experience with DABs, dbt, Terraform, and Kafka is highly valued.

Qualifications

• Strong hands-on Data BAU operations and data freshness monitoring.
• Experience with ITIL and ITSM processes including incident, problem, change, and release management.
• Experience with ServiceNow or Jira Service Management for SLA tracking and RCA documentation.

Responsibilities

• Design, develop, optimize, and support enterprise-grade data engineering solutions.
• Maintain production data platforms and monitor pipelines for quality and SLAs.
• Resolve production incidents and ensure data quality across systems.
• Collaborate on cloud-based data platforms using Databricks and AWS.
• Implement automated CI/CD and IaC for data workflows.

Skills

Data BAU operations

ITIL/ITSM

Databricks

AWS

Spark tuning

DABs (Databricks Asset Bundles)

dbt

Kafka streaming

Terraform

GitLab CI/CD

Unix/Linux scripting

Tools

AWS S3

Databricks

Delta Live Tables

Unity Catalog

Liquid Clustering

Z‑Ordering

Spark optimization

Terraform

GitLab

dbt Core/Cloud

Kafka

Job description

Senior Data Engineer (Databricks)

Experience Level: 7+ Years
Job Type: Full-time, Permanent
Location: Client Site (On-site – Brisbane)

Experience Level: 7+ Years
Job Type: Full-time, Permanent
Location: Client Site (On-site – Brisbane)

Role Overview

We are seeking an experienced Senior Data Engineer with strong hands‑on expertise in Databricks and AWS to design, develop, optimize, and support enterprise‑grade data engineering solutions. The ideal candidate will have extensive experience in building scalable data pipelines, implementing modern data transformation frameworks, and maintaining reliable cloud‑based data platforms.

The role combines platform engineering with production Data Business As Usual (BAU) support, requiring strong capabilities in monitoring data pipelines, resolving production incidents, maintaining data quality, and ensuring compliance with operational SLAs. A solid understanding of ITIL and IT Service Management (ITSM) practices is essential for maintaining platform stability and driving continuous improvement across production data environments.

Technical Competencies & Requirements

1. Data BAU, ITIL & ITSM (Critical)

• Strong hands‑on experience in Data BAU operations, supporting production data pipelines, monitoring data freshness, and resolving data quality alerts.

• Thorough understanding of ITIL frameworks and ITSM processes, including Incident, Problem, Change, and Release Management.

• Experience with ITSM tools such as ServiceNow or Jira Service Management for SLA tracking, incident management, escalation handling, and Root Cause Analysis (RCA) documentation.

2. AWS & Databricks Ecosystem

• Hands‑on experience with AWS cloud services, particularly Amazon S3.

• Advanced proficiency in Databricks development using PySpark, including Unity Catalog, Delta Live Tables (DLT)/Lakeflow Declarative Pipelines, Serverless Compute, SQL Warehouses, dashboards, and alerting.

• Strong understanding of performance optimization techniques, including Liquid Clustering, Z‑Ordering, and Spark query optimization.

• Practical experience managing deployments and workloads using Databricks Asset Bundles (DABs).

3. Data Ecosystem & Infrastructure

• Advanced experience with dbt (dbt Core or dbt Cloud) for developing and managing version‑controlled data transformations.

• Hands‑on experience building and supporting Kafka‑based streaming pipelines for real‑time and near‑real‑time data integration.

• Proven experience with Terraform for Infrastructure as Code (IaC), including provisioning and managing AWS and Databricks resources.

• Strong experience with GitLab for source control and implementing automated GitLab CI/CD pipelines.

• Good working knowledge of Unix/Linux environments, including scripting, troubleshooting, and operational support.

Preferred Qualifications

• Experience working in the Airline or Aviation domain will be highly regarded.

• Databricks certifications will be an added advantage.