Data Engineer

PRIMUS Global Solutions (PRIMUS UK & Europe); Sydney; On-site; AUD 120,000 - 180,000; Full time

Job summary

PRIMUS Global Solutions (PRIMUS UK & Europe) is seeking an experienced Snowflake + dbt Engineer to design, develop, and maintain scalable data transformation pipelines using modern cloud data technologies. The role focuses on strong SQL, dbt Core, Snowflake, and Airflow expertise to ensure data accuracy and pipeline performance.

You will build Python-based data ingestion processes, implement CI/CD with GitHub Actions, and work with AWS services to deliver reliable, business-ready datasets with

Qualifications

• Strong SQL skills and hands-on experience with Snowflake.
• Proficiency in dbt Core for data transformation and modeling.
• Experience implementing incremental loads, deduplication, and data transformation logic.
• Hands-on experience developing and debugging Apache Airflow DAGs.
• Working knowledge of AWS data and orchestration services.
• Experience with GitHub Actions, CI/CD, and version control using Git.
• Knowledge of data quality testing, YAML configurations, and source-to-target mapping documentation.
• Experience handling structured or fixed-format financial data files.

Responsibilities

• Design, develop, and maintain data transformation pipelines using Snowflake, dbt Core, and SQL.
• Implement incremental data loads, deduplication, and efficient data transformation strategies in Snowflake.
• Develop, maintain, and troubleshoot Apache Airflow DAGs for data pipeline orchestration.
• Build Python-based data ingestion processes to extract, process, and load data from multiple sources.
• Utilize AWS services, including S3, Lambda, ECS, MWAA, Glue Data Catalog, Lake Formation, SQS, and Event Bridge.
• Implement CI/CD pipelines using GitHub Actions to automate testing and deployment workflows.
• Develop data quality tests using YAML configurations and maintain source-to-target mapping documentation.
• Process structured and fixed-format financial data files, ensuring data integrity and consistency.
• Monitor pipeline performance, troubleshoot failures, and optimize data processing workflows.
• Collaborate with data engineers, analysts, and business stakeholders to deliver reliable datasets.

Skills

SQL proficiency

Analytical thinking

Problem solving

Communication

Collaboration

Independent work

Tools

Snowflake

dbt Core

Apache Airflow

Python

AWS

GitHub Actions

Yaml configurations

Job description

We are looking for an experienced Snowflake + dbt Engineer to design, develop, and maintain scalable data transformation pipelines using modern cloud data technologies. The ideal candidate will have strong SQL and dbt Core expertise, hands-on experience with Snowflake and Apache Airflow, and a solid understanding of AWS services and Python-based data ingestion.

The role involves building reliable, high-quality, business-ready datasets while ensuring data accuracy, pipeline performance, and efficient CI/CD processes.

Key Responsibilities

• Design, develop, and maintain data transformation pipelines using Snowflake, dbt Core, and SQL.
• Implement incremental data loads, deduplication logic, and efficient data transformation strategies in Snowflake.
• Develop, maintain, and troubleshoot Apache Airflow DAGs for data pipeline orchestration.
• Build Python-based data ingestion processes to extract, process, and load data from multiple sources.
• Utilize AWS services, including S3, Lambda, ECS, MWAA, Glue Data Catalog, Lake Formation, SQS, and Event Bridge.
• Implement CI/CD pipelines using GitHub Actions to automate testing and deployment workflows.
• Develop data quality tests using YAML configurations and maintain source-to-target mapping documentation.
• Process structured and fixed-format financial data files, ensuring data integrity and consistency.
• Monitor pipeline performance, troubleshoot failures, and optimize data processing workflows.
• Collaborate with data engineers, analysts, and business stakeholders to deliver reliable and well-documented datasets.

Mandatory Skills

• Strong SQL skills and hands-on experience with Snowflake.
• Proficiency in dbt Core for data transformation and modeling.
• Experience implementing incremental loads, deduplication, and data transformation logic.
• Hands-on experience developing and debugging Apache Airflow DAGs.
• Working knowledge of AWS data and orchestration services.
• Experience with GitHub Actions, CI/CD, and version control using Git.
• Knowledge of data quality testing, YAML configurations, and source-to-target mapping documentation.
• Experience handling structured or fixed-format financial data files.

Good-to-Have Skills

• Experience with AWS Managed Workflows for Apache Airflow (MWAA).
• Familiarity with AWS Glue Data Catalog and Lake Formation.
• Experience with event-driven architectures using SQS and Event Bridge.
• Understanding of data warehouse design, data modeling, and pipeline optimization.
• Experience working with financial or other business-critical datasets.
• Strong analytical and problem-solving skills.
• Attention to data quality, accuracy, and reliability.
• Ability to troubleshoot complex data pipeline issues.Effective communication and collaboration skills.
• Ability to work independently in a contract-based delivery environment.