Data Engineer

Africa Practice; Johannesburg; Hybrid; ZAR 900,000 - 1,300,000; Full time

Benefits offered by this job

Competitive remuneration

Growth opportunities

Diverse and experienced team

Job summary

Africa Practice is seeking a Data Engineer to build and optimise data pipelines within a hybrid work model across Africa. You will integrate open-source tools with our GCP environment to enhance scalability while supporting analysts and data scientists in producing high-value insights for clients.

The role requires strong SQL/Python skills, experience with BigQuery and data lakehouse concepts, and a collaborative approach with internal stakeholders.

Qualifications

• Bachelor's degree in CS/IT/engineering or related quantitative field.
• Strong hands-on experience with cloud data engineering using GCP services.
• Deep proficiency with open-source data tools: Airflow, dbt, Spark or Kafka.
• Solid understanding of data modelling, ETL/ELT, and data lakehouse concepts.

Responsibilities

• Build, maintain, and optimise data pipelines and ETL/ELT processes.
• Integrate open-source tools into GCP infrastructure for scalability.
• Manage data storage, schemas and query performance (BigQuery, Cloud Storage).
• Support data scientists by preparing datasets for analytics and ML models.
• Create and monitor cloud jobs on GCP for daily ingestions.

Skills

SQL

Python

Data modelling

ETL/ELT development

GCP data services

Data storage & schemas

DevOps & CI/CD

AI coding tools

Education

Bachelor's degree in computer science or related field

Tools

Google BigQuery

Cloud Storage

Airflow

dbt

Apache Spark

Kafka

Git / GitHub Actions

Docker

Job description

Data Engineer

Contract: Full-time; 2-year fixed term, renewable

Remuneration: Competitive, based on experience and competence

Location: Any AP office location, though South Africa is preferred; hybrid

Reports to: Partner, Product Innovation

About us

Africa Practice is a mission-led strategic consulting firm advising investors and development partners across the continent. Our mission is to shape a prosperous Africa that inspires the world. For over 20 years, we have been partnering with global investors, policy-makers, development partners and other changemakers to drive transformation on the continent.

Our insights and advocacy solutions are rooted in detailed systems and stakeholder understanding. We identify, generate, and analyse human-centred systems data to design impactful advocacy interventions. We deliver our world-class insights and advocacy solutions through our local presence.

We operate as one company, with a diverse team working on a hybrid basis in Abuja, Cape Town, Dakar, Dar es Salaam, Gaborone, Johannesburg, Lagos, Nairobi, and London. We are risk analysts, development economists, advocates, storytellers, data scientists, policy experts, systems thinkers and creative minds. Over 80% of our staff are African and over 50% are women. We have been a signatory to the UN Global Compact since 2021 and achieved BCorp certification in 2024.

About the role

This is a technical, hands-on role requiring strong execution in data pipeline engineering, database management, and system optimisation. The successful candidate will support the evolution of our data infrastructure by building, maintaining, and expanding our hybrid data lakehouse.

You will focus on integrating robust open-source applications alongside our existing Google Cloud Platform (GCP) environment to enhance scalability and flexibility. Working closely with our analytics and consulting teams, you will ensure our data assets are reliable, accessible, and secure, directly enabling the delivery of superior insights to our clients.

Responsibilities:

Data lakehouse development and optimisation:

• Build, maintain, and optimise reliable data pipelines and ETL/ELT processes to ingest data from diverse networks and sources.
• Integrate open-source tools and frameworks into our existing GCP infrastructure to enhance pipeline efficiency and flexibility.
• Manage and optimise data storage, schemas, and query performance within Google BigQuery and cloud storage systems (Cloud SQL, Amazon RDS).
• Support data scientists and analysts by structuring, cleaning, and preparing datasets for advanced analytics and machine learning models.
• Capable of creating, deploying, logging, monitoring and maintaining cloud jobs on GCP for daily data ingestions

Data governance and compliance:

• Implement and adhere to robust data governance policies, ensuring all data engineering activities comply with relevant regulations (e.g. GDPR, POPIA) and ethical best practices.
• Build data quality checks, monitoring systems, and testing frameworks to safeguard the accuracy and reliability of our data pipelines.
• Document data architectures, workflows, and schemas to support internal data literacy and maintain the company's technical knowledge base.

Operational support:

• Monitor daily data operations, troubleshooting pipeline failures, handling data ingestion issues, and resolving system bottlenecks promptly.
• Support the cost-efficient use of cloud resources and open-source deployments through continuous performance tuning.

Stakeholder engagement:

• Collaborate closely with internal stakeholders (consultants, analysts, and data scientists) to understand their data requirements and translate them into efficient technical solutions.
• Communicate technical data pipeline concepts and limitations clearly to non-technical team members.

What we are looking for:

• Proven experience working as a data engineer or in a similar technical data infrastructure role, preferably within a professional services or fast-paced consulting environment.
• Strong hands-on experience with cloud technologies, specifically Google Cloud Platform (GCP) services relevant to data handling (e.g. BigQuery, Cloud Storage).
• Deep technical proficiency with open-source data engineering tools, particularly for orchestration (e.g. Apache Airflow), transformation (e.g. dbt), and processing (e.g. Apache Spark or Kafka).
• Solid understanding of data modelling concepts, ETL/ELT development, and data lakehouse architectures.
• High proficiency in SQL and Python for data manipulation and pipeline construction.
• Experience with DevOps and CI/CD tools, including Git, GitHub Actions, and Docker containerisation.
• Be AI Native - have considerable experience using AI coding tools such as Codex, Claude, Cursor
• Familiarity with handling both structured and unstructured data streams.
• A bachelor's degree in computer science, information technology, engineering, or a related quantitative field.

What we offer

We offer the successful candidate an opportunity to develop a rewarding career within a dynamic pan-African consultancy advising a prestigious portfolio of clients. This position offers:

• Competitive remuneration
• Immediate opportunities for growth
• Collaboration with a diverse and experienced team of colleagues

Don’t meet every single requirement? Studies have shown that women and people of colour are much less likely to apply for a position unless they meet every single qualification. So this notwithstanding, if you are excited about this role, we would love to hear from you. At Africa Practice, we are dedicated to building an inclusive, diverse and authentic workplace. You might just be the new team member we are looking for - for this or other roles.

Do you want to contribute to an influential and prosperous Africa, inspiring the world? We’re looking for Africa’s changemakers – smart people with strong analytical, advocacy, digital and problem-solving skills who can reconcile diverse perspectives to deliver sustainable change.

The vacancy is immediately open for suitable applications and will be closed as soon as a suitable candidate is identified. Early applications are advised.