Developer II | Retail IT | BI

Allan Gray; Cape Town; On-site; ZAR 650,000 - 900,000; Full time

Job summary

Allan-Gray is seeking a Developer II for Retail IT BI to modernise the data platform across Snowflake and SQL Server. You will design data pipelines, optimize ELT workloads, and contribute to data quality, testing, and observability while collaborating with senior engineers on platform modernization.

You will implement CI/CD, IaC, and robust monitoring while supporting production data platforms and incident resolution in a dynamic environment.

Qualifications

• 3-5 years of experience in data engineering, warehousing, or BI development.
• Strong SQL skills and complex query troubleshooting.
• Experience building ETL/ELT pipelines and data integration.
• Familiarity with dimensional modelling and data warehousing concepts.
• Experience with large datasets, cloud platforms (AWS or Snowflake), and CI/CD.
• Knowledge of Python, MSSQL or similar data tech stack; Agile methodology.

Responsibilities

• Design, develop, and maintain data pipelines using Snowflake and SQL Server.
• Build and optimize ELT workloads, data models, and transformations.
• Create Dagster assets, schedules, sensors, and orchestration workflows.
• Implement data warehousing dimensional models for operational and analytic use cases.
• Troubleshoot data quality, pipelines, and platform issues across Snowflake and SQL Server.
• Analyze query performance, warehouse usage, and storage strategies.
• Develop automated testing, reconciliation, and data validation mechanisms.
• Contribute to CI/CD, deployment automation, and source control standards.
• Implement monitoring, observability, and operational controls for availability.
• Collaborate with architects to modernize legacy ETL and data platform architecture.
• Participate in code reviews and technical design discussions; document procedures.

Skills

SQL

ETL/ELT

Snowflake

Python

Dagster

Cloud AWS

Git

CI/CD

Infrastructure as Code

Monitoring

Automated testing

MS SQL Server

Education

Degree or diploma in information science

Tools

Snowflake

MS SQL Server

Dagster

Python

SSIS/SSRS/SSAS

Git

Job description

Career Opportunities: Developer II | Retail IT | BI (9174)

Join the Retail IT Data Platform and Warehousing Development team in modernising and evolving both our Microsoft SQL Server-based data warehouse and Snowflake-based analytical platform. The role contributes to the development, support, and evolution of data products and operational processes across both environments, while helping establish modern DataOps practices.

Job Responsibilities

• Design, develop, and maintain data pipelines and data products using Snowflake, Microsoft SQL Server and MSSQL Data tools (SSIS, SSRS, SSAS), Python, and Dagster.
• Develop and optimize ELT workloads, stored procedures, data models, and transformation logic to support analytical and operational reporting requirements.
• Build and maintain Dagster assets, schedules, sensors, and orchestration workflows to ensure reliable and observable data processing.
• Design and implement data models for both operational and analytical use cases, applying appropriate warehousing and dimensional modelling techniques.
• Troubleshoot and resolve data quality, pipeline, orchestration, and platform issues across Snowflake and SQL Server environments.
• Analyze and optimize query performance, pipeline execution, warehouse utilization, and storage strategies.
• Develop automated testing, reconciliation, and data validation mechanisms to improve platform reliability and data quality.
• Contribute to CI/CD pipelines, deployment automation, source control standards, and engineering best practices.
• Implement monitoring, alerting, observability, and operational controls to improve platform availability and supportability.
• Collaborate with architects and senior engineers to modernize legacy ETL processes and evolve the team's data platform architecture.
• Conduct code reviews and contribute to technical design discussions, ensuring solutions align with team standards, security requirements, and architectural principles.
• Create and maintain technical documentation, platform runbooks, operational procedures, and support artefacts.
• Investigate emerging technologies and contribute to technical spikes and proof-of-concepts that improve platform capabilities.
• Provide production support for critical data platforms, participating in incident diagnosis, root-cause analysis, and resolution activities

Requirements

• 3-5 years' experience in data engineering, data warehousing, BI development, or a related field.
• Strong SQL skills with experience developing and troubleshooting complex queries.
• Experience building and supporting ETL/ELT pipelines.
• Understanding of dimensional modelling and data warehousing concepts.
• Experience working with large datasets and data integration processes.
• Exposure to cloud platforms such as AWS or Snowflake.
• Experience with Git-based source control and development workflows.
• Understanding of CI/CD concepts and deployment automation.
• Familiarity with Infrastructure-as-Code concepts.
• Experience supporting production data platforms and resolving operational issues.
• Understanding of monitoring, logging, and observability principles.
• Exposure to automated testing and data quality practices.
• Strong troubleshooting and analytical skills.
• Experience with Python, SQL, MSSQL or similar data engineering technologies.
• Understanding of software development lifecycle practices.
• Familiarity with Agile delivery methodologies.

Skills and Competencies

• Attention to detail and quality of work.
• Ability to take initiative and responsibility.
• Strong analytical skills.
• Good communication
• Provide accurate estimates
• Present solutions clearly
• Ability to work well under pressure and meet deadlines without sacrificing quality.

Education

• Degree or diploma in information science or related technical discipline
• Knowledge of financial concepts and terms and interested in attending courses to improve business knowledge
• Financial Services industry experience would be an advantage