Lead Data Engineer

Placements24; Paarl; On-site; ZAR 1,200,000 - 2,000,000; Full time

Benefits offered by this job

Competitive salary

Medical aid

Retirement fund

Career advancement opportunities

Job summary

Placements24 in Paarl, Western Cape seeks a Lead Data Engineer to build and manage scalable data pipelines, warehousing, and data lakes. You will lead a team to ensure reliable ingestion, transformation, and storage for BI and advanced analytics.

The role requires a strong background in ETL/ELT, SQL, and big data tech, with leadership experience to mentor engineers and drive data strategy in a dynamic analytics firm.

Qualifications

• Bachelor's degree or higher in CS/Engineering or quantitative field.

Responsibilities

• Lead design, development, and maintenance of robust data pipelines (ETL/ELT).
• Architect and implement data warehousing and data lake solutions.
• Manage and optimize cloud-based data infrastructure (AWS/Azure/GCP).
• Ensure data quality, integrity, and security across systems.
• Develop data modeling best practices and governance.
• Collaborate with data scientists, analysts, and stakeholders to capture data requirements.
• Mentor and guide junior data engineers; promote best practices.
• Automate data processes and monitor pipeline performance.
• Evaluate and adopt new data technologies and tools.
• Troubleshoot and resolve data infrastructure issues.

Skills

Leadership

ETL design

SQL

Python/Scala

Big data technologies

Education

Bachelor's degree in CS/Engineering

Tools

Spark

Hadoop

Airflow

Luigi

AWS Redshift

S3

Glue

Azure Data Factory

BigQuery

Dataflow

Job description

About the Role

Our client, a growing data analytics firm, is looking for a highly skilled Lead Data Engineer to build and manage their data infrastructure. This role is essential for developing scalable and reliable data pipelines that support business intelligence and advanced analytics. You will lead a team of data engineers, architecting solutions that ensure efficient data ingestion, transformation, and storage. The ideal candidate has a strong background in data warehousing, ETL processes, and big data technologies. This is a pivotal role for driving data strategy and innovation in Paarl, Western Cape .

Key Responsibilities

• Lead the design, development, and maintenance of robust and scalable data pipelines (ETL/ELT).
• Architect and implement data warehousing solutions and data lakes to support business needs.
• Manage and optimize data infrastructure on cloud platforms (e.g., AWS, Azure, GCP).
• Ensure data quality, integrity, and security across all data systems.
• Develop and enforce data modeling best practices.
• Collaborate with data scientists, analysts, and business stakeholders to understand data requirements.
• Mentor and guide junior data engineers, fostering best practices and technical excellence.
• Automate data processes and monitor data pipeline performance.
• Evaluate and implement new data technologies and tools.
• Troubleshoot and resolve issues related to data infrastructure and pipelines.

Requirements

• Bachelor's degree in Computer Science, Engineering, or a related quantitative field.
• Minimum of 8 years of experience in data engineering, with at least 3 years in a leadership role.
• Proven experience designing and implementing complex ETL/ELT pipelines.
• Strong expertise in SQL and data warehousing concepts.
• Hands-on experience with big data technologies (e.g., Spark, Hadoop ecosystem).
• Proficiency in programming languages such as Python or Scala.
• Experience with cloud data services (e.g., AWS Redshift, S3, Glue; Azure Data Factory; Google BigQuery, Dataflow).
• Familiarity with data orchestration tools (e.g., Airflow, Luigi).
• Excellent problem-solving, analytical, and communication skills.
• Leadership experience and ability to mentor a team.

Benefits

Our client offers a competitive salary, comprehensive medical aid, retirement fund, and opportunities for career advancement. You will be part of a dynamic team working with cutting-edge data technologies. This role is based in Paarl, Western Cape and offers a chance to lead significant data initiatives.