MS Fabric Data Engineer (JHB/CPT)

DataFin; Cape Town, Johannesburg; On-site; ZAR 800,000 - 1,200,000; Full time

Job summary

A tailored financial solutions provider in Cape Town is seeking an experienced MS Fabric Data Engineer. The successful candidate will lead a data team, design scalable data pipelines, and manage the data warehouse. Ideal applicants should have over 5 years of experience with Azure Data Technologies and must hold relevant Microsoft certifications. This role provides a chance to work on innovative data solutions while mentoring a dynamic team.

Qualifications

• 5+ years of experience working with Azure Data Technologies.
• Proficiency in T-SQL, ETL/ELT tools, PySpark, and Spark SQL.
• Strong experience in leading and managing a Data team.

Responsibilities

• Lead the design and maintenance of scalable data pipelines in Microsoft Fabric.
• Maintain and update reports and dashboards in Power BI.
• Oversee data ingestion, transformation, and integration processes.

Skills

Azure Data Technologies

T-SQL

ETL/ELT tools

PySpark

Spark SQL

Power BI

Data governance

Education

Microsoft Azure Data Engineer Associate certification

Microsoft Fabric Data Engineer Associate certification

Tools

Azure Synapse

Azure Data Factory

Azure Databricks

Azure SQL

Power BI

Job description

We are seeking a passionate leader for a MS Fabric Data Engineer role with a provider of tailored Financial Solutions. The successful candidate will lead the initial phases of a warehouse rework project and support the ongoing expansion of a newly created data warehouse within the Microsoft Fabric platform. They will also build and lead a Data team, playing a critical role in designing, implementing, and maintaining data pipelines and ensuring high-quality data movement.

Responsibilities:

• Maintain and update existing reports and dashboards in Power BI to reflect evolving business requirements.
• Assist with data extractions and execute defined SQL queries.
• Perform data validation and reconciliations between reporting layers.
• Work closely with the Analytics and Engineering teams to resolve data discrepancies.
• Support documentation of reports, dashboards, and data definitions.
• Participate in data testing and user feedback processes.
• Lead the design, development, and maintenance of scalable data pipelines and infrastructure within Microsoft Fabric.
• Architect and optimize the data warehouse within the Microsoft Fabric platform.
• Oversee data ingestion, transformation, and integration processes to ensure efficient data workflows.
• Implement and maintain ETL/ELT processes, data governance, and security best practices.
• Work closely with the Analyst and business stakeholders to translate requirements into technical solutions.
• Develop the data model to be implemented to achieve the business’ analytics objectives.
• Ensure data quality, reliability, and performance across all pipelines.
• Build and lead a Data team, initially managing the existing Analyst and gradually hiring additional Engineers and Analysts as needed.
• Provide mentorship, training, and career development opportunities for team members.
• Stay updated on emerging technologies in data engineering, analytics, and Microsoft Fabric advancements.

Requirements:

Preferred Qualifications include one of the following Microsoft Certifications: Azure Data Engineer Associate (DP-203), Fabric Analytics Engineer Associate (DP-600), or Fabric Data Engineer Associate (DP-700). The ideal candidate should have 5+ years of experience working with complimentary Azure Data Technologies and proficiency in T-SQL, ETL/ELT tools, PySpark, and Spark SQL. Strong experience in leading and managing a Data team is also required.

Experience/Skills:

• 5+ years of experience working with Azure Data Technologies - Azure Synapse, Azure Data Factory, Azure Databricks, Azure SQL, etc.
• Designing and developing Star Schema based data models.
• Designing and implementing analytics solutions using data warehouses and data lakes.
• Hands-on experience with ETL/ELT tools, data modelling, and performance optimization.
• Understanding of data governance, security, and compliance best practices.
• Proven experience in using Pipelines, Lake House, and Warehouse capabilities of the Microsoft Fabric platform.
• Strong experience in developing Semantic Models (datasets) in Power BI.
• Proficiency in T-SQL in SQL Server using Stored Procedures and Functions.
• Proficiency in PySpark and Spark SQL within notebooks on the Microsoft Fabric Platform or Azure Databricks.
• Experience with Azure DevOps and GitHub for CI/CD deployments and use of Power BI Deployment Pipelines.

Leadership & Business Skills:

• Experience leading and managing a Data team, with a track record of hiring and mentoring data professionals.
• Strong ability to translate business requirements into technical solutions.
• Excellent communication skills to engage with stakeholders across business and technical teams.
• Experience in Project Management, Agile methodologies, and process automation.

Advantageous:

• Experience with Machine Learning models or AI-driven analytics.
• Background in Financial Services, Sales Optimization, or Regulatory Reporting.
• Experience with Premium or Embedded.