DBT Data Engineer

Tech Aalto; Sydney; Presencial; AUD 90 000 - 130 000; Tempo integral

Resumo da oferta

Tech Aalto Pte Ltd in Sydney, Australia seeks a data engineer to build and maintain dbt transformation models across end-to-end data pipelines. You will write Snowflake SQL for complex transformations and develop Airflow DAGs on AWS ECS.

The role involves Python-based ingestion tasks, Lambda functions, and data quality tests. You will maintain lineage docs, support data sharing to downstream consumers, and own tickets from design to production.

Qualificações

• Experience with data transformation and data warehousing.
• Strong knowledge of dbt Core and Snowflake SQL.
• Experience developing DAGs with Apache Airflow.
• Proficiency in Python for data ingestion and validation.
• Hands-on AWS services (S3, Lambda, MWAA, ECS, Glue Catalogs, Lake Formation).
• Exposure to event-driven data sharing and SQS/EventBridge.
• Experience in CI/CD using GitHub Actions.
• Familiarity with structured financial or fixed-format data files is a plus.

Responsabilidades

• Build and maintain dbt transformation models across multiple pipeline layers from raw ingestion to governed datasets.
• Write Snowflake SQL for complex transformations with type casting, deduplication, incremental load logic, surrogate keys.
• Develop and debug Apache Airflow DAGs on AWS ECS.
• Contribute to AWS Lambda-based ingestion (Python) including file validation, control file generation, and S3 event handling.
• Maintain source-to-target mapping documentation and align models to business requirements and data specs.
• Support onboarding of downstream consumers via data sharing mechanisms and event-driven notifications.
• Participate in code reviews, write YAML-based data quality tests, and own tickets from design through production.

Ferramentas

Snowflake SQL

Python

Apache Airflow

AWS

dbt Core

S3

MWAA

ECS

Glue Catalogs

Lake Formation

Lambda

SQS

EventBridge

GitHub Actions

Iceberg

Descrição da oferta de emprego

• Building and maintaining dbt transformation models across multiple pipeline layers — from raw ingestion through to governed, consumer-ready datasets
• Writing Snowflake SQL for complex transformations: type casting, deduplication, incremental load logic, surrogate key generation
• Developing and debugging Apache Airflow DAGs on AWS ECS
• Contributing to AWS Lambda-based ingestion( Python) — file validation, control file generation, S3 event handling
• Maintaining source-to-target mapping documentation and aligning models to business requirements and data specs
• Supporting onboarding of downstream consumers via data sharing mechanisms and event-driven notifications
• Participating in code reviews, writing YAML-based data quality tests, and owning tickets from design through to production
• Proficiency with dbt Core (Must have)
• Apache Iceberg Table format (Nice to have)
• AWS - Working Knowledge of S3, Lambda, MWAA, ECS, Glue Catalogs, Lake Formation, Lambda, SQS, EventBridge
• Experience with fixed-format or structured financial data files
• CICD with Github Actions.

Confidentiality is assured, and only shortlisted candidates will be notified for interviews.