Senior Data & AI Engineer

Cloudbox; Cape Town; On-site; ZAR 1,000,000 - 1,800,000; Full time

Job summary

Cloudbox in Cape Town is building a unified data and reporting platform on Microsoft Fabric. You will own the medallion lakehouse end-to-end—from vendor data ingestion through the gold layer to an AI-assisted reporting surface—and drive it as the long‑term platform owner.

This greenfield build requires senior expertise in Fabric, OneLake, dimensional modelling, and production data pipelines from multiple APIs.

Qualifications

• Senior-level experience with Microsoft Fabric, OneLake, or equivalent modern lakehouse platforms.
• Strong dimensional modelling and semantic modelling background.
• Experience building production data pipelines from third-party/vendor APIs.
• Hands-on with the Azure services surrounding Fabric (Function Apps, Key Vault, Blob Storage).
• Source control and CI/CD experience with Azure DevOps and GitHub.
• Security-first engineering habits and secrets management (Key Vault).
• Familiarity with governance, lineage, and sensitivity labelling (Purview).
• Knowledge of grounding techniques for LLM-based querying over structured data.

Responsibilities

• Own the data platform end-to-end from ingestion to a query-ready gold layer and AI-assisted reporting surface.
• Design the medallion architecture (bronze/silver/gold) and implement the ingestion pipelines.
• Develop the semantic model serving dashboards with row-level security.
• Build AI proof-of-concept with provenance and anti-fabrication controls.
• Ensure data quality, observability, and reconciliations across the pipeline.
• Governance with Purview across medallion layers.
• Document and transfer knowledge to the wider team.

Skills

Microsoft Fabric

OneLake

Dimensional modelling

Semantic models

Data pipelines

Vendor APIs

Security and Key Vault

Purview governance

LLM grounding (RAG)

Tools

Azure DevOps

GitHub

Bicep

Fabric Git integration

Job description

About the role

Cloudbox is building a unified multi-tenant data and reporting platform on Microsoft Fabric, aggregating 10+ vendor APIs (Autotask PSA, Microsoft 365/Graph, Azure Monitor, Microsoft Defender, Huntress, Proofpoint, Cato SASE, BambooHR, Xero, and internal systems) into a medallion lakehouse for hedge fund and financial-services clients. You'll own the data platform end-to-end, from raw vendor ingestion through to a query-ready gold layer and an AI-assisted reporting layer. This is a greenfield build: you will design the medallion architecture, build it, and then run it as its long-term owner.

What you'll own

• Metadata-driven OneLake medallion lakehouse architecture (bronze/silver/gold)


• Azure Functions-based vendor ingestion pipelines across the full vendor set, fanned out per tenant from a central metadata registry, starting with an Autotask-first vertical slice


• Dimensional modelling and semantic model design - the semantic layer serving dashboards and reporting, with row-level security (RLS) for client-level access


• The AI proof-of-concept - grounded, provenance-backed natural-language querying over platform data, with anti-fabrication controls


• Data quality, reconciliation, and observability for the pipeline layer - structured logging, SLOs, and alerting, with reconciliation proven against synthetic vendor data before production


• Data governance with Microsoft Purview - lineage, classification, and sensitivity labelling across the medallion layers


• Documentation and knowledge transfer - runbooks, architecture decision records, and pairing with the wider team, so the platform never carries a bus factor of one

What we're looking for

• Senior-level experience with Microsoft Fabric, OneLake, or equivalent modern lakehouse platforms (Databricks, Synapse acceptable if willing to specialise into Fabric)


• Strong dimensional modelling and semantic model design background (star schemas, conformed dimensions, Fabric/Power BI semantic models)


• Experience building production data pipelines from third-party/vendor APIs - ideally within a metadata-driven ingestion framework


• Hands-on with the Azure services surrounding Fabric - Function Apps, Key Vault, Blob Storage, and Static Web Apps


• Source control and CI/CD experience with both Azure DevOps and GitHub, including Fabric Git integration; Bicep or equivalent infrastructure-as-code a plus


• Security-first engineering habits - managed identities and Key Vault for all secrets; no credentials in code, ever


• Working knowledge of Microsoft Purview (or equivalent) for governance, lineage, and sensitivity labelling - this platform serves regulated financial-services clients


• Familiarity with grounding techniques for LLM-based querying over structured data (RAG, citation/provenance patterns) is a plus - the core screen for this role is lakehouse and modelling depth, and the AI layer can be developed in-role with specialist support


• Comfortable being the technical anchor for the platform's most critical dependency chain

Nice to have:

Exposure to Azure AI Foundry (model deployment, evaluation, prompt flow), Fabric IQ and its ontology layer, Fabric data agents, MCP-style agent tool-calling, Azure Key Vault / Managed HSM and customer-managed key (CMK) patterns, and Fabric capacity planning (F-SKU sizing) - these support later platform tiers and none is a screening requirement