Principal Data Engineer

Placements24; Stellenbosch; Hybrid; ZAR 1,500,000 - 2,500,000; Full time

Benefits offered by this job

Health insurance

Retirement contributions

Life assurance

Professional development budget

Conference access

Collaborative culture

Stellenbosch office

Job summary

Placements24 is seeking a Principal Data Engineer for a hybrid role in Stellenbosch, Western Cape. You will architect, design, and optimize large-scale data pipelines and data platforms to enable advanced analytics and ML initiatives.

You will lead a team, mentor engineers, shape data governance, collaborate with data science and business teams, and evaluate new technologies to keep our data architecture robust, scalable, and cost-efficient.

Qualifications

• Master’s or Ph.D. in a quantitative field.
• 10+ years of data engineering experience.
• Minimum of 5 years in a lead or principal engineering role.
• Extensive experience with cloud data services (AWS/Azure/GCP).
• Proficiency in Python, Scala, or Java.
• Deep understanding of distributed systems and big data technologies (Spark, Hadoop).
• Strong experience with SQL and NoSQL databases.
• Expertise in data warehousing concepts, ETL/ELT processes, and data modeling.

Responsibilities

• Design and maintain scalable data pipelines for batch and real-time data processing.
• Architect and implement data warehousing and data lakes on cloud platforms (AWS/Azure/GCP).
• Optimize data processing workflows for performance, cost, and reliability using Spark, Hadoop, Kafka.
• Develop and enforce data governance policies and standards for data quality, security, and compliance.
• Collaborate with data scientists, analysts, and stakeholders to understand data requirements and deliver tailored solutions.
• Implement data modeling techniques to support BI and analytics.
• Troubleshoot and resolve complex data infrastructure issues.
• Provide technical leadership and mentorship to a team of data engineers.
• Evaluate and recommend new technologies and tools to enhance data infrastructure capabilities.
• Ensure scalability and efficiency of data storage and processing.

Skills

Technical leadership

Data governance

Cloud platforms

Mentorship

Communication

Education

Master's or PhD in Computer Science/Engineering/Mathematics

Tools

Spark

Hadoop

Kafka

Python

Scala

Java

AWS

Azure

BigQuery

Job description

About the Role

Our client, a leading financial services institution, is seeking a highly experienced and visionary Principal Data Engineer to join their advanced analytics division. This hybrid role is based in the vibrant tech hub of Stellenbosch, Western Cape . You will be responsible for architecting, building, and optimizing our client's data pipelines and infrastructure, enabling sophisticated data analysis and machine learning initiatives. As a Principal Engineer, you will provide technical leadership, mentor junior team members, and drive innovation in data architecture and processing. The ideal candidate possesses a deep understanding of big data technologies, cloud platforms, and data warehousing principles, coupled with a strong commitment to data quality and governance.

Key Responsibilities

• Design, develop, and maintain scalable and robust data pipelines for batch and real-time data processing.
• Architect and implement data warehousing solutions and data lakes on cloud platforms (e.g., AWS, Azure, GCP).
• Optimize data processing workflows for performance, cost, and reliability using big data technologies like Spark, Hadoop, and Kafka.
• Develop and enforce data governance policies and standards to ensure data quality, security, and compliance.
• Collaborate with data scientists, analysts, and business stakeholders to understand data requirements and deliver tailored solutions.
• Implement data modeling techniques to support business intelligence and analytical reporting needs.
• Troubleshoot and resolve complex data infrastructure issues.
• Provide technical leadership and mentorship to a team of data engineers.
• Evaluate and recommend new technologies and tools to enhance data infrastructure capabilities.
• Ensure the scalability and efficiency of data storage and processing solutions.

Requirements

• Master's or Ph.D. in Computer Science, Engineering, Mathematics, or a related quantitative field.
• 10+ years of experience in data engineering, with a significant focus on building and managing large-scale data platforms.
• Minimum of 5 years of experience in a lead or principal engineering role.
• Extensive experience with cloud data services (e.g., AWS Redshift/S3/EMR, Azure Data Factory/Synapse, GCP BigQuery/Dataflow).
• Proficiency in programming languages such as Python, Scala, or Java.
• Deep understanding of distributed systems and big data technologies (Spark, Hadoop ecosystem).
• Strong experience with SQL and NoSQL databases.
• Expertise in data warehousing concepts, ETL/ELT processes, and data modeling.
• Experience with data streaming technologies (e.g., Kafka, Kinesis) is highly desirable.
• Excellent leadership, communication, and problem-solving skills.
• Ability to work effectively in a hybrid work environment.

Benefits

Our client offers an industry-leading salary, attractive bonuses, and a comprehensive benefits package including health insurance, retirement contributions, and life assurance. Employees benefit from generous professional development budgets, opportunities to attend leading conferences, and a collaborative work environment within our modern Stellenbosch office. This role offers a unique opportunity to shape the data landscape of a major financial institution and drive significant business impact.