Azure Data Engineer

Ariston; Randburg; On-site; ZAR 700,000 - 1,000,000; Full time

Job summary

Ariston is seeking an experienced Azure Data Engineer to design, build, and maintain data pipelines and warehouse structures that underpin an enterprise analytics platform operating in a regulated, high‑volume environment.

This hands‑on role requires delivering scalable pipelines, complex pricing logic, and a robust Power BI model, with collaboration across data architects, analysts, and business stakeholders to translate requirements into reliable data solutions.

Qualifications

• Bachelor's degree or equivalent demonstrable work experience.
• Five+ years commercial data engineering, with at least three on the Microsoft data platform.
• Experience designing and implementing production data pipelines using Azure Data Factory.
• Advanced SQL and experience with SQL Server/Azure SQL Database.
• Strong data modelling, warehousing principles, and dimensional modelling.

Responsibilities

• Design, develop, and maintain scalable data pipelines using Azure Data Factory to extract, transform, and load data into the data warehouse.
• Implement and maintain complex business calculation logic within the data layer, ensuring it is traceable to documented business requirements and auditable.
• Model and maintain warehouse structures that support effective-dated pricing and rate tables, versioned business rules, and multi-level classification hierarchies.
• Collaborate with data architects, analysts, data scientists, and stakeholders to translate requirements into technical solutions.
• Build and maintain data structures that feed the Power BI semantic layer and support reporting grains.
• Implement data transformation and cleansing logic to ensure data quality, accuracy, and consistency across the pipeline.
• Write and execute unit tests for all components and document results.
• Participate in peer code reviews before code promotion.
• Support System Integration Testing and defect triage with timely remediation.
• Optimize pipelines for performance, scalability, and cost efficiency.
• Monitor pipeline health and troubleshoot failures with corrective actions.
• Develop technical documentation, including design specs and mappings.
• Provide guidance to junior team members and transfer knowledge across the team.
• Adhere to security and compliance standards to protect data privacy.
• Stay current with the Microsoft data platform roadmap and assess new capabilities.

Skills

Analytical thinking

Problem solving

Communication

Team collaboration

Attention to detail

Education

Bachelor's degree in Computer Science / Information Systems / Engineering

Tools

Azure Data Factory

Azure Data Lake Storage

Azure Synapse Analytics

Power BI

SQL Server / Azure SQL Database

Python

C#

Job description

As a digital strategy, delivery, and assurance company that attracts innovative, energetic, and driven individuals. We pride ourselves on making a positive and lasting impact on our clients through our commitment to excellence. As professionals, we are energised by the opportunity to solve complex business challenges, leveraging the power of emerging digital technologies.

We are seeking an experienced Azure Data Engineer to join our delivery team on a permanent basis. You will design, build, and maintain the data pipelines and warehouse structures that underpin an enterprise analytics and forecasting platform operating in a regulated, high-volume commercial environment.

This is a hands‑on engineering role on a live, business-critical system. The platform carries substantial embedded business logic — layered pricing and tariff structures, multi‑tier customer classification, and calculation rules that change in response to external regulatory events. Getting the data model and the transformation logic right is the job. Candidates who have only built generic ingestion pipelines are unlikely to find this role a comfortable fit.

Key Responsibilities

• Design, develop, and maintain scalable data pipelines using Azure Data Factory to extract, transform, and load data from source systems into the data warehouse.
• Implement and maintain complex business calculation logic within the data layer, ensuring it remains traceable to documented business requirements and can be evidenced under audit.
• Model and maintain warehouse structures that support effective‑dated pricing and rate tables, versioned business rules, and multi‑level classification hierarchies.
• Collaborate with data architects, functional analysts, data scientists, and business stakeholders to translate documented requirements into technical solutions.
• Build and maintain the data structures that feed the Power BI semantic layer, working alongside the reporting team to ensure the model supports required reporting grains.
• Implement data transformation and cleansing logic to ensure data quality, accuracy, and consistency across the pipeline.
• Write and execute unit tests for all developed components before handover, and record the results as a delivery artefact.
• Participate in mandatory peer code review, both as author and reviewer, prior to any code being promoted.
• Support System Integration Testing, including defect triage, root cause analysis, and remediation within agreed turnaround times.
• Optimise pipelines for performance, scalability, and cost efficiency.
• Monitor pipeline health, troubleshoot failures, and implement corrective and preventative measures.
• Develop and maintain technical documentation, including design specifications, source‑to‑target mappings, transformation rules, and workflow diagrams.
• Provide technical guidance to junior team members and contribute to knowledge transfer across the team.
• Adhere to security and compliance standards, ensuring data privacy and confidentiality across all data processing activities.
• Stay current with the Microsoft data platform roadmap and assess the applicability of new capabilities to our delivery estate.

Requirements

Essential Domain Experience

This is the differentiating requirement for this role.

You must be able to demonstrate prior delivery on a data warehouse or analytics platform carrying substantial embedded commercial logic. Concretely, we are looking for hands‑on experience with several of the following:

• Pricing, tariff, or rate structures where the applicable rate varies by customer category, time period, seasonal window, consumption or usage band, or a combination of these.
• Multi‑level customer classification hierarchies where the classification itself determines which calculation path applies, and where reclassification has downstream consequences for historical reporting.
• Effective‑dated reference data and slowly changing dimensions, including the handling of retrospective rate changes and back‑dated adjustments.
• Externally driven restructures — regulatory, legislative, or commercial — that force wholesale change to pricing dimensions and cascade through the model, the calculation logic, and every downstream report.
• Reconciliation of derived or forecast figures against actuals at more than one level of aggregation, where the two must tie out and any variance must be explainable.
• High‑volume transactional or consumption data aggregated to a reporting grain, where the choice of grain is itself a modelling decision with commercial consequences.

This experience is typically gained in utilities and energy, telecommunications, financial services, insurance, or any environment operating a billing, rating, or revenue management platform. We are interested in the shape of the problem you have solved, not the sector it sat in.

Qualifications and Experience

• Bachelor's degree in Computer Science, Information Systems, Engineering, or a related field; or equivalent demonstrable work experience.
• A minimum of five years' commercial data engineering experience, of which at least three are on the Microsoft data platform.
• Demonstrated experience designing and implementing production data pipelines using Azure Data Factory.
• Advanced SQL, including performance tuning, and experience with relational databases such as SQL Server and Azure SQL Database.
• Strong grounding in data modelling, data warehousing principles, and dimensional modelling techniques.
• Working knowledge of the Azure data services estate, including Azure Data Lake Storage, Azure Synapse Analytics, and Microsoft Fabric.
• Practical familiarity with Power BI, including an understanding of how warehouse design decisions constrain what the semantic model and DAX layer can do.
• Experience with data integration patterns and technologies, including REST APIs, file‑based interfaces, and message queues.
• Solid programming skills in Python or C# for scripting and automation.
• Experience working within a formal SDLC, including source control, release management, and structured testing cycles.
• Excellent analytical and problem‑solving skills, with the ability to troubleshoot complex data issues and performance bottlenecks.
• Strong written and verbal communication skills, with the ability to engage both technical and non‑technical stakeholders.

Certifications

Current Microsoft certification is a requirement for this role. Candidates who are not yet certified but who meet the experience criteria may be considered on the basis of a committed certification timeline, which we will fund and support.

Required — at least one of:

• Microsoft Certified: Fabric Data Engineer Associate (DP-700).
• Microsoft Certified: Azure Data Engineer Associate (DP-203). Note that this certification was retired on 31 March 2025 and can no longer be earned or renewed; it will be accepted as evidence of prior competence, but holders will be expected to progress to DP-700.

Advantageous:

• Microsoft Certified: Power BI Data Analyst Associate (PL-300).
• Microsoft Certified: Fabric Analytics Engineer Associate (DP-600).
• Microsoft Certified: Azure Solutions Architect Expert (AZ-305).
• Microsoft Certified: Azure Database Administrator Associate (DP-300).

Attributes We Look For

• You test your own work before anyone else sees it, and you can show what you tested.
• You read the requirement document rather than inferring the requirement from the ticket title.
• You raise the problem early, while it is still cheap to fix.
• You are comfortable being the person who owns a component end to end.
• You write documentation that someone else could actually pick up.