Microsoft Fabric Specialist / Data Engineer

Praesignis

South Africa

Hybrid

ZAR 900,000 - 1,300,000

Full time

3 days ago

Be an early applicant

Application generator

A complete application in a minute — tailored resume and cover letter, ready to send.

Get past ATS filters

Benefits offered by this job

Hybrid work model

Professional development

Medical aid

Retirement benefits

Career growth in AI/data tech

Job summary

Praesignis seeks an experienced AI Data Engineer to build and maintain scalable data pipelines powering AI initiatives in Soweto. You will design, implement, and optimize ETL/ELT processes, ensure data quality and security, and collaborate with data scientists and ML engineers to deliver reliable data services.

The role emphasizes multi-tenant data isolation, governance and integration with REST/SOAP APIs, SQL, Python and PySpark, using Microsoft Fabric and cloud platforms.

Qualifications

Bachelor's degree or equivalent in a quantitative field.
3+ years of data engineering experience with big data.
Proficient in Python/SQL/Scala and ETL/ELT.
Experience with Spark/Hadoop and cloud data services.
Strong data modeling and architecture knowledge.
Azure Fabric and/or Azure data certifications preferred.
Hands-on with Microsoft Fabric components (OneLake, Lakehouse, Warehouse, Data Pipelines, notebooks).
CI/CD with Git/Azure DevOps and data governance awareness.
Power BI support experience and data reporting.
Excellent documentation and stakeholder communication skills.

Responsibilities

Design and build scalable ETL/ELT pipelines for AI/ML workloads.
Maintain data quality, security, and availability across platforms.
Develop Data Platform APIs for ingestion, processing, and access.
Collaborate with data scientists and engineers to meet data requirements.
Monitor pipelines, resolve failures, and optimize performance.
Ensure multi-tenant data isolation and proper governance.

Skills

3+ years experience

Python

SQL

Scala

data modeling

data architecture

Power BI

data transformation

Education

Bachelor's degree in CS/Engineering/Statistics

Microsoft Fabric Azure certification

Tools

Spark

Hadoop

Azure

OneLake

Lakehouse

Data Factory

SQL Server/T-SQL

REST/SOAP APIs

Job description

About the Role

An established insurance company is seeking to hire a highly skilled and experienced Data Engineer to join their team. The role is critical for building and maintaining the robust data infrastructure that powers AI and machine learning initiatives. The ideal candidate possesses a strong background in data engineering principles, experience with big data technologies, and a passion for enabling data-driven innovation through reliable and efficient data systems.

About the Role

Our client is seeking a meticulous and experienced AI Data Engineer to join their team in Soweto. This role is critical for building and maintaining the robust data infrastructure that powers our AI and machine learning initiatives. You will be responsible for designing, developing, and optimizing data pipelines, ensuring data quality, availability, and accessibility for data scientists and ML engineers.

Key Responsibilities

Design and build scalable and reliable data pipelines for AI/ML workloads.
Develop ETL/ELT processes to ingest, transform, and load data from various sources.
Ensure data quality, integrity, and security across all data platforms.
Collaborate with data scientists and ML engineers to understand data requirements and provide solutions.
Optimize data infrastructure for performance, cost, and scalability.
Implement data governance policies and best practices.

Responsibilities

Build and maintain ETL pipelines supporting a multi-tenant data platform, ingesting data from APIs, databases, and event sources.
Build and maintain Data Platform APIs that allow teams to ingest, process, and access data easily and reliably.
Implemented tenant-specific logic by following existing configuration and naming conventions.
Apply tenant-level data isolation using schemas, partitions, or access controls.
Build models from existing templates used for financial and operational reporting. Develop models for analytics and reporting, maintaining consistency with shared data models.
Monitor scheduled pipelines, investigate failures, and resolve data quality issues and inconsistencies.
Maintain daily and incremental data loads into the data warehouse.
Assist with onboarding new clients by validating source data and testing pipeline outputs.
Work closely with senior data engineers to learn patterns for multi-tenant data isolation.
Collaborate with analytics, product, and customer facing teams to understand reporting needs.
Support strict regulatory and audit requirements by following data handling, retention, and audit guidelines.
Handle financial and sensitive data (PII) according to company policies and regulatory standards (e.g., POPIA).
Apply least-privilege access and role‑based access controls, and support data protection through masking, encryption, and established security standards.

Requirements

Bachelor's degree in Computer Science, Engineering, Statistics, or a related quantitative field.
3+ years of experience in data engineering, with a focus on big data technologies.
Proficiency in programming languages such as Python, SQL, or Scala.
Experience with big data platforms like Spark, Hadoop, or similar.
Familiarity with cloud data services (AWS, Azure, GCP) and data warehousing concepts.
Strong understanding of data modeling, database design, and data architecture.
Relevant Microsoft Fabric and/or Azure data certification.
Strong hands‑on experience with Microsoft Fabric, including OneLake, Lakehouse, Warehouse, Data Pipelines/Data Factory, notebooks and Dataflows Gen2.
Strong SQL Server and T‑SQL capability, including complex query development, schema design, indexing and performance optimisation.
Practical experience developing, maintaining and supporting production ETL/ELT pipelines.
Experience integrating and extracting data from REST/SOAP APIs, databases, flat files and other structured or unstructured data sources.
Proficiency in data transformation using SQL and Python/PySpark, with an understanding of scalable data processing practices.
Practical experience in data warehousing, dimensional modelling, incremental loading, orchestration and schema evolution.
Experience with troubleshooting pipeline failures, data quality issues and performance bottlenecks, including the ability to restore service efficiently.
Experience with source control, CI/CD and deployment practices using Git, Azure DevOps or equivalent tools.
Experience supporting Power BI and other downstream analytical or reporting requirements.
Demonstrated ability to take ownership of an existing technical environment with limited hand‑holding.
Strong documentation, communication and stakeholder engagement skills.

Benefits

Competitive salary and comprehensive benefits package.
Opportunities for professional growth and training in AI and data technologies.
Access to modern data infrastructure and cutting‑edge tools.
A collaborative and innovative work environment in Soweto.
Hybrid work model offering flexibility.

Benefits

Competitive salary and performance-based bonuses.
Comprehensive medical aid and retirement benefits.
Opportunities for professional development and training in cutting‑edge data technologies.
Hybrid work model offering flexibility and team collaboration.
Contribute to critical data infrastructure that powers advanced AI capabilities.

Benefits