Data Engineering Lead

Blue Pearl PTY; Johannesburg; On-site; ZAR 1,200,000 - 1,900,000; Full time

Job summary

Blue Pearl PTY in Johannesburg, South Africa, seeks an experienced Data Engineering Lead to design, build, and lead enterprise-scale data platforms enabling BI, analytics, reporting, and data-driven decision-making. The role combines hands-on leadership with technical depth across cloud data platforms, ETL/ELT, data architecture, and SQL optimisation.

You will provide technical leadership across the full data engineering lifecycle, mentoring engineers, driving standards, and collaborating with

Qualifications

• Bachelor's degree or higher in a relevant field and postgraduate qualifications advantageous.
• 8–10+ years' experience in Data Engineering with 3+ years leading data engineering teams.
• Proven experience building scalable ETL/ELT solutions for enterprise environments.
• Experience supporting reporting, analytics, and data science initiatives.
• Experience implementing data governance frameworks and data quality controls.

Responsibilities

• Lead the design, development, and continuous improvement of enterprise-grade data platforms.
• Architect scalable data pipelines for batch and near real-time processing.
• Mentor junior and intermediate engineers and drive engineering best practices.
• Enforce enterprise data engineering standards, governance, and metadata management.
• Collaborate with business stakeholders, BI teams, and governance teams to deliver analytics-ready data assets.
• Optimize cloud-native architectures for performance, cost, and reliability.
• Develop robust ingestion, storage, and data integration components.

Skills

Technical Leadership

Stakeholder Management

Mentoring

Analytical Thinking

Planning and Organising

Education

Bachelor's Degree in Computer Science

Bachelor's Degree in Information Systems

Bachelor's Degree in Information Technology

Bachelor's Degree in Software Engineering

Bachelor's Degree in Data Science

Tools

Snowflake

Databricks

Airflow

AWS

Terraform

Power BI

Qlik Sense

Tableau

SQL Server

Job description

Johannesburg, South Africa | Posted on 07/14/2026

Our client is seeking an experienced Data Engineering Lead to design, build, and lead enterprise‑scale data platforms that enable business intelligence, advanced analytics, reporting, and data‑driven decision‑making.

The successful candidate will provide technical leadership across the full data engineering lifecycle, ensuring scalable, secure, reliable, and high‑performing data solutions. This role requires close collaboration with business stakeholders, solution architects, BI teams, governance specialists, and software engineering teams to deliver trusted and analytics‑ready data assets.

The ideal candidate is a hands‑on technical leader with extensive experience in cloud data platforms, ETL/ELT development, data architecture, SQL optimisation, and modern data engineering practices.

Key Responsibilities

• Data Engineering Leadership
• Lead the design, development, implementation, and continuous improvement of enterprise‑grade data engineering solutions.
• Architect scalable, secure, reusable, and high‑performance data pipelines supporting batch and near real‑time processing.
• Provide technical leadership across multiple data engineering initiatives and mentor junior and intermediate engineers.
• Establish and enforce enterprise data engineering standards, frameworks, and best practices.
• Drive innovation and continuous improvement across the organisation's data platform capabilities.
• Design robust ingestion frameworks for structured, semi‑structured, and unstructured data.
• Develop scalable storage architectures supporting data warehousing and lake‑house environments.
• Implement efficient ETL and ELT processing frameworks.
• Optimise cloud‑native data architectures for performance, scalability, reliability, and cost efficiency.
• Design resilient solutions supporting analytics, reporting, AI, and machine learning initiatives.
• Build, maintain, and optimise enterprise data pipelines.
• Develop complex SQL queries, stored procedures, and transformation logic.
• Design and maintain reusable data integration components.
• Perform data transformation, cleansing, enrichment, and validation processes.
• Ensure consistent data availability and integrity across enterprise platforms.

Data Governance & Quality

• Embed data quality controls throughout the engineering lifecycle.
• Implement automated validation, monitoring, lineage, metadata management, and observability.
• Ensure compliance with organisational governance frameworks and regulatory requirements.
• Secure sensitive and confidential information through appropriate engineering controls.
• Maintain technical documentation including source‑to‑target mappings and technical specifications.

Stakeholder Management

• Collaborate closely with business stakeholders, business intelligence teams, data analysts, solution architects, application development teams, governance teams, and infrastructure teams.
• Translate business requirements into scalable technical solutions.
• Communicate technical concepts clearly to both technical and non‑technical audiences.
• Provide regular project updates, technical recommendations, and risk assessments.

Operational Excellence

• Troubleshoot pipeline failures and performance issues.
• Optimise processing times and infrastructure costs.
• Lead root‑cause analysis and incident resolution.
• Support production deployments and platform upgrades.

Requirements

Education

• Bachelor's Degree in Computer Science
• Bachelor's Degree in Information Systems
• Bachelor's Degree in Information Technology
• Bachelor's Degree in Software Engineering
• Bachelor's Degree in Data Science
• Bachelor's Degree in Mathematics
• Bachelor's Degree in Statistics
• Or a related discipline

Relevant postgraduate qualifications will be advantageous.

Experience

• 8–10+ years' experience in Data Engineering.
• Minimum 3 years leading technical data engineering teams.
• Proven experience building scalable ETL/ELT solutions.
• Experience working within enterprise environments.
• Experience supporting reporting, analytics, and data science initiatives.
• Experience implementing data governance frameworks.
• Experience in Financial Services, Mining, Retail, Telecommunications, or Consulting environments will be advantageous.

Technical Knowledge

• Modern Data Engineering architectures
• Data Warehousing
• Lake house Architecture
• ETL / ELT Frameworks
• Data Integration
• Cloud Computing
• SQL Performance Optimisation
• Metadata Management
• Data Governance
• Master Data Management
• Data Quality
• CI/CD
• Infrastructure as Code
• Software Development Lifecycle (SDLC)
• Information Security
• Data Privacy Regulations

Technical Skills

• AWS
• Snowflake
• Databricks
• Advanced SQL
• Python
• Spark
• PySpark
• AWS Glue
• Airflow
• Power BI
• Qlik Sense
• QlikView
• Tableau
• SQL Server
• Oracle
• Amazon Redshift
• Git
• Terraform
• API Integration
• REST Services
• Data Security
• Data Encryption

Desired Attributes

• Technical Leadership
• Strategic Thinking
• Analytical Thinking
• Strong Problem‑Solving Ability
• Stakeholder Management
• Decision‑Making Ability
• Accountability
• Attention to Detail
• Results Orientation
• Planning and Organising
• Mentoring and Coaching

Preferred Certifications

• Microsoft Azure Solutions Architect Expert
• AWS Certified Solutions Architect – Associate / Professional
• Databricks
• Snowflake
• Certified Data Management Professional (CDMP)
• TOGAF (Advantageous)

Key Success Measures

• Delivery of scalable, secure data platforms.
• Data pipeline reliability and availability.
• Data quality and integrity.
• Platform performance and optimisation.
• Successful project delivery within agreed timelines.
• Reduction in operational incidents.
• Engineering automation and efficiency improvements.
• Compliance with enterprise governance standards.
• Mentoring and development of engineering team members.

Why Join This Opportunity?

This is an exciting opportunity to lead the development of modern enterprise data platforms within a collaborative and innovative environment. You will play a key role in shaping the organisation's data engineering capability, enabling business intelligence, advanced analytics, AI initiatives, and digital transformation through scalable, secure, and high‑performing data solutions.