Data Platform Engineer II: Retail IT & BI

Allan Gray; Cape Town; On-site; ZAR 650,000 - 900,000; Full time

Job summary

Allan-Gray is seeking a Developer II for Retail IT BI to modernise the data platform across Snowflake and SQL Server. You will design data pipelines, optimize ELT workloads, and contribute to data quality, testing, and observability while collaborating with senior engineers on platform modernization.

You will implement CI/CD, IaC, and robust monitoring while supporting production data platforms and incident resolution in a dynamic environment.

Qualifications

• 3-5 years of experience in data engineering, warehousing, or BI development.
• Strong SQL skills and complex query troubleshooting.
• Experience building ETL/ELT pipelines and data integration.
• Familiarity with dimensional modelling and data warehousing concepts.
• Experience with large datasets, cloud platforms (AWS or Snowflake), and CI/CD.
• Knowledge of Python, MSSQL or similar data tech stack; Agile methodology.

Responsibilities

• Design, develop, and maintain data pipelines using Snowflake and SQL Server.
• Build and optimize ELT workloads, data models, and transformations.
• Create Dagster assets, schedules, sensors, and orchestration workflows.
• Implement data warehousing dimensional models for operational and analytic use cases.
• Troubleshoot data quality, pipelines, and platform issues across Snowflake and SQL Server.
• Analyze query performance, warehouse usage, and storage strategies.
• Develop automated testing, reconciliation, and data validation mechanisms.
• Contribute to CI/CD, deployment automation, and source control standards.
• Implement monitoring, observability, and operational controls for availability.
• Collaborate with architects to modernize legacy ETL and data platform architecture.
• Participate in code reviews and technical design discussions; document procedures.

Skills

SQL

ETL/ELT

Snowflake

Python

Dagster

Cloud AWS

Git

CI/CD

Infrastructure as Code

Monitoring

Automated testing

MS SQL Server

Education

Degree or diploma in information science

Tools

Snowflake

MS SQL Server

Dagster

Python

SSIS/SSRS/SSAS

Git

Job description

Allan-Gray is seeking a Developer II for Retail IT BI to modernise the data platform across Snowflake and SQL Server. You will design data pipelines, optimize ELT workloads, and contribute to data quality, testing, and observability while collaborating with senior engineers on platform modernization.

You will implement CI/CD, IaC, and robust monitoring while supporting production data platforms and incident resolution in a dynamic environment.