Senior Data Engineering Technical Lead

Network1; Cape Town; On-site; ZAR 800 000 - 1 200 000; Full time

Job summary

A technology recruitment firm is seeking a Senior Data Engineering Technical Lead in Cape Town. This role involves solving complex data problems, leading migrations, and building high-performance platforms. Ideal candidates have 8+ years in data engineering, expertise in Scala, and experience with cloud platforms. The position offers full-time employment with a focus on delivering impactful data solutions. Join a dynamic team to achieve real business outcomes through innovative data engineering.

Qualifications

• Minimum 8 years experience in data engineering or related roles.
• Proven expertise in large-scale data migration and ETL pipeline development.
• Production experience with cloud data platforms.

Responsibilities

• Lead data migration strategies across multi-petabyte datasets.
• Mentor engineers through pair programming and code reviews.
• Ensure performance and system health of data pipelines.

Skills

Data engineering experience

Proficiency in Scala and Java

Apache Spark expertise

Distributed systems design skills

Cloud platforms experience

Education

Bachelor's or Master's degree in Computer Science or Engineering

Tools

Apache Spark

Amazon Web Services

Google Cloud Platform

Microsoft Azure

Databricks

Job description

Overview

Were looking for a Senior Data Engineering Technical Lead who thrives on solving complex problems at scale. This role offers the opportunity to lead enterprise-grade data initiatives, design and build high-performance data platforms, and work hands-on with petabyte-scale systems in modern cloud environments.

You\'ll combine deep technical expertise with leadership, guiding engineers, shaping architecture, and delivering mission-critical data solutions that power real business outcomes. If you\'re passionate about data engineering ownership and building systems that truly matter, this is the role for you.

Key Responsibilities

• Data Engineering & Migration Leadership:
  
  
  
  • Design and deliver large-scale data migration strategies supporting identity transformation across multi-petabyte datasets
  
  
  • Develop robust data lineage mapping and validation frameworks to ensure accuracy and integrity
  
  
  • Build scalable batch and streaming data transformation pipelines with parallel orchestration
  
  
  • Lead JDK and Apache Spark upgrades while maintaining production stability
  
  
  • Implement monitoring and observability to ensure data pipeline performance and system health
  
  
  • Create comprehensive testing and validation frameworks for complex migration scenarios
  
  
  
  


• Technical Leadership & Engineering Excellence:
  
  
  
  • Own and provide technical leadership for critical system components
  
  
  • Mentor engineers through pair programming, code reviews, and collaborative delivery
  
  
  • Lead technical design discussions aligned with enterprise standards
  
  
  • Drive delivery momentum by proactively identifying and removing blockers
  
  
  • Collaborate cross-functionally to ensure strong technical alignment and successful outcomes
  
  
  
  

Job Experience and Skills Required

• Minimum 8 years experience in data engineering or related engineering roles


• Bachelor\'s or Master\'s degree in Computer Science, Engineering, or equivalent practical experience


• Proven expertise in large-scale data migration and ETL pipeline development


• Advanced proficiency in Scala and Java using Gradle or Maven


• Deep experience with Apache Spark and distributed storage systems (HDFS, S3, and GCS)


• Production experience with cloud data platforms including Amazon Web Services, Google Cloud Platform, Microsoft Azure, or Databricks


• Hands-on experience with Amazon EKS and containerised workloads


• Strong understanding of data privacy, security, and compliance principles (GDPR preferred)


• Excellent distributed systems design, analytical, and problem-solving skills

Apply now!

Additional Information

• For more exciting IT vacancies please visit: also specialise in recruiting in the following:
  
  
  
  • Software Developers
  
  
  • Mobile Developers
  
  
  • Business and Systems Analysts
  
  
  • BI and SQL Experts
  
  
  
  


• If you have not had any response in two weeks please consider the vacancy application unsuccessful. Your profile will be kept on our database for any other suitable roles/positions.

For more information contact:

Lameez Mohamed
Specialist Recruitment Consultant
Connect with me on LinkedIn!

Experience: Senior IC

Key Skills
Administrative Skills, Facilities Management, Biotechnology, Creative Production, Design And Estimation, Architecture

Employment Type : Full-Time

Experience: years

Vacancy: 1