Senior Data Engineer: Build & Govern Scalable Data Pipelines

PRIMUS Global Solutions (PRIMUS UK & Europe); Sydney; Sur place; AUD 120 000 - 160 000; Plein temps

Résumé du poste

PRIMUS Global Solutions is seeking a data engineer in Sydney to build and maintain dbt transformation models across pipelines, from raw data to governed, consumer-ready datasets. You will write Snowflake SQL for complex transformations and develop Apache Airflow DAGs on AWS ECS.

You will also contribute to Python-based ingestion using AWS Lambda, validate files, and handle S3 events, while aligning models to business data specs and enabling downstream data sharing.

Qualifications

• Strong SQL proficiency required.
• Experience with dbt Core.
• Experience writing Snowflake SQL for complex transformations.
• Experience developing and debugging Apache Airflow DAGs.
• Familiarity with AWS services (S3, Lambda, MWAA, ECS, Glue Catalogs, Lake Formation, Lambda, SQS, EventBridge).
• Experience with fixed-format or structured financial data files.

Responsabilités

• Building and maintaining dbt transformation models across multiple pipeline layers — from raw ingestion through to governed, consumer-ready datasets
• Writing Snowflake SQL for complex transformations: type casting, deduplication, incremental load logic, surrogate key generation
• Developing and debugging Apache Airflow DAGs on AWS ECS
• Contributing to AWS Lambda-based ingestion( Python) — file validation, control file generation, S3 event handling
• Maintaining source-to-target mapping documentation and aligning models to business requirements and data specs
• Supporting onboarding of downstream consumers via data sharing mechanisms and event-driven notifications
• Participating in code reviews, writing YAML-based data quality tests, and owning tickets from design through to production
• Strong SQL
• Proficiency with dbt Core
• Apache Iceberg Table format (Nice to have)
• AWS - Working Knowledge of S3, Lambda, MWAA, ECS, Glue Catalogs, Lake Formation, Lambda, SQS, EventBridge
• Experience with fixed-format or structured financial data files

Connaissances

Strong SQL

Outils

dbt Core

Snowflake SQL

Apache Airflow

Python

AWS Lambda

S3

ECS

MWAA

Glue Catalogs

Lake Formation

SQS

EventBridge

Description du poste

PRIMUS Global Solutions is seeking a data engineer in Sydney to build and maintain dbt transformation models across pipelines, from raw data to governed, consumer-ready datasets. You will write Snowflake SQL for complex transformations and develop Apache Airflow DAGs on AWS ECS.

You will also contribute to Python-based ingestion using AWS Lambda, validate files, and handle S3 events, while aligning models to business data specs and enabling downstream data sharing.