Data Principal Engineer

Techrepo.co.za; Cape Town; On-site; ZAR 1,200,000 - 1,800,000; Full time

Job summary

Techrepo.co.za is seeking a Senior Data Architect to lead the design of a scalable data platform, aligning AI/ML workflows with governance standards. You will drive architecture, supervise engineers, and spearhead data quality initiatives across logistics and e-commerce domains.

The role requires deep GCP, data modelling, and real-time streaming expertise, with a focus on reducing technical debt and standardising technologies across teams. Cape Town-based position with potential for hybrid work.

Qualifications

• Bachelor's degree in CS, Engineering, or IS; equivalent experience considered.
• 8+ years in data engineering; 3+ years at principal/architect level.
• Track record delivering large-scale data platforms (lakehouse/warehouse, real-time streaming).
• Experience leading platform simplification and reducing technical debt.
• Hands-on with AI/ML data infrastructure and AI tooling integration.
• GCP expertise with BigQuery and Dataform; Looker/LookML knowledge.

Responsibilities

• Ecosystem Architecture & Platform Simplification: create and maintain a master data-architecture blueprint.
• Real-Time Operational Enablement: build live data layer for operational analytics across the Group.
• Data Governance Standards & Documentation: define standards, lineage, and data-management practices.
• AI Enablement & Automation: define guardrails, data contracts, and AI tool integration patterns.
• Technical Leadership & Mentorship: act as senior contributor, mentor engineers, and set standards.

Skills

Data architecture

Leadership

Python

SQL

Data modelling

AI/ML data infra

Education

Bachelor's degree in Computer Science / Engineering / Information Systems

Tools

BigQuery

Dataform

Looker/LookML

Terraform

Kafka / Pub/Sub

dbt

Job description

Job Description

About the Role

We are a young, dynamic, hyper-growth company looking for smart, creative, hard-working people with integrity to join us!

Responsibilities

• Ecosystem Architecture & Platform Simplification:
• Produce a definitive, up-to-date master blueprint of the Group's data architecture, sources, flows, models, KPI mappings and use it to drive a structured simplification programme; cutting over-engineering and technical debt, standardising technology choices across teams, and making it faster to deliver new data products.
• Real-Time Operational Enablement:
• Logistics, Supply Chain, and Distribution Centre operations need faster access to operational data than our current batch-oriented warehouse provides.
• Design and build the event-driven, live data layer that decouples these systems from the historical reporting warehouse, enabling faster and more precise operational analytics across the Group.
• Data Governance Standards & Documentation:
• Establish and own the technical standards that underpin the Group's data governance programme.
• Defining data quality standards, lineage documentation requirements, and data management practices, and building them into sprint workflows as normal engineering practice.
• Working with central team SMEs (Data Engineering, Analytics Engineering, BI, DataOps) to turn existing engineering practice into formal, Group-level domain standards.
• Building and maintaining a centralised architecture repository as the Group's single source of truth for how data flows across the ecosystem. Given the scale involved, hundreds of systems across 15-20+ business units. This is a phased build: the first 90 days should produce the repository's structure and the first few highest-priority domains, with full coverage growing over the following quarters.
• Keeping standards and documentation current as the platform evolves.
• AI Enablement, Integration & Automation Platform:
• Define the technical guardrails that let teams innovate safely: data contracts, security and privacy controls, model governance patterns, and clear standards for embedding automation into data engineering operations.
• Own platform alignment for AI consumption, so BigQuery, Dataform, and Looker expose data that AI tools and copilots can use reliably and safely: documented schemas, semantic layers, data contracts, consistent access patterns.
• Lead integration planning for AI tooling on the platform, secure, well-governed connection patterns for AI agents, in line with the Group's AI Data Policy.
• Design and build automation that keeps governance and platform operations manageable as the Group scales, automated maturity telemetry from platform metadata (classification tag coverage, lineage completeness, Dataform test coverage, Looker documentation completeness), self-service onboarding tooling, and AI-assisted copilots that cut manual facilitation work across the Group.
• Technical Onboarding, Training & Knowledge Transfer:
• Replace fragmented, course-heavy onboarding with a structured learning path and reference architecture guides tailored to different skill levels (engineering, analytics, BI).
• Design specialist technical training modules covering architecture and standards, and contribute content into the Group's tiered governance training.
• Success here means the team can operate without depending on any one person's knowledge.
• Technical Leadership & Mentorship:
• As the highest individual contributor in the data domain, you weigh in on design disagreements, unblock the most complex cross-functional challenges, and set the engineering standard for the division.
• Where a technical recommendation conflicts with a team's delivery priorities, the accountable manager makes the final call, your job is to bring the strongest technical case to that decision.
• You coach and mentor senior and staff-level engineers, advise the Engineering Director on platform and governance strategy, and contribute to the technical career path matrix for the department.
• Minimum Required Qualification:
• A Bachelor's degree in Computer Science, Engineering, Information Systems, or a related field is preferred; equivalent demonstrated experience at this scale and seniority will be considered in place of formal qualifications.
• Minimum Required Experience:
• 8+ years experience in data engineering, including at least 3 years at principal or architect level in a complex, high-scale environment.
• A track record designing and delivering large-scale data platforms: lakehouse/warehouse, real-time event streaming, data ingestion frameworks.
• Experience leading platform simplification or technical debt reduction, not just greenfield builds.
• Experience contributing technical standards into a formal data governance programme (DMBOK familiarity a plus).
• Hands‑on experience with AI or ML data infrastructure: feature stores, model serving pipelines, data contracts, drift monitoring.
• Experience integrating AI/LLM tooling (copilots, agents, RAG) with a data platform, access patterns, semantic layers, and data contracts that let AI tools consume data safely.
• A track record of building automation or internal tooling, dashboards, scripts, copilots that cuts manual operational work.
• Exposure to logistics, e-commerce, or supply chain data is a plus.
• Deep GCP expertise, particularly BigQuery and Dataform.
• Strong experience with stream processing frameworks (Kafka, Pub/Sub, or equivalent).
• Solid command of dimensional and event-driven data modelling; working knowledge of Looker/LookML.
• Experience with Infrastructure as Code (Terraform or equivalent) and CI/CD for data pipelines.
• Strong Python and SQL; familiarity with dbt-style transformation frameworks and orchestration tools.
• Understanding of POPIA obligations as they apply to data processing and governance.
• Can produce clear architecture documentation: data flow diagrams, ADRs, technical blueprints, onboarding guides.
• Experience designing for auditability, data lineage tracing, and compliance requirements.
• Comfortable weighing build vs. buy trade‑offs and driving technology standardisation across teams.
• Comfortable acting as the final technical authority in a data domain — setting standards, making calls, bringing teams along.
• Experience advising senior stakeholders and translating technical trade-offs into business terms.
• Genuinely invested in growing the people around you, through mentorship and knowledge-sharing.
• Pragmatic: understands the best architecture is the one the team can actually run.