Senior Data Engineer

Finance Corporate; Cape Town; On-site; ZAR 700 000 - 900 000; Full time

Job summary

A client is seeking a highly skilled Senior Data Engineer in Cape Town. The role involves building scalable systems, optimizing data pipelines, and utilizing AWS technologies. Ideal candidates will have extensive experience in data engineering and a strong understanding of data modeling.

Qualifications

• 5 years of experience in data engineering or software engineering.
• 2 years of experience with big data technologies.
• AWS Certification preferred.

Responsibilities

• Architect and build highly scalable distributed systems.
• Develop and optimize ETL processes for data pipelines.
• Collaborate with teams to define data engineering requirements.

Skills

Data Engineering

Python

AWS

Data Modelling

Big Data

Education

Bachelors Degree in Computer Science

Tools

Apache Hive

Hadoop

Spark

Talend

Kafka

Job description

A client is looking for a highly skilled and experienced Senior Data Engineer to join their team based in Cape Town. The ideal candidate will have extensive experience in data engineering big data technologies cloud platforms (particularly AWS) and a deep understanding of data modelling. This is an exciting opportunity to work on largescale data infrastructure projects and drive innovation within the company.

Duties & Responsibilities

Key Responsibilities :

• Architect and build highly scalable distributed systems using opensource tools.
• Design and implement efficient data models with a deep understanding of various data structures and their benefits and limitations.
• Work with big data batch and streaming tools to process and analyze data at scale.
• Develop and optimize Extract Transform Load (ETL) processes for data pipelines.
• Utilize AWS technologies (EMR EC2 and S3 for data storage and processing.
• Write and maintain clean efficient code in Python PySpark or Spark.
• Collaborate with crossfunctional teams to define data engineering requirements and deliver technical solutions.
• Ensure the smooth operation performance and scalability of data infrastructures.
• Continuously improve the reliability and security of data systems.

Requirements :

• Bachelors Degree in Computer Science Computer Engineering or a related field or equivalent experience.
• AWS Certification (e.g. AWS Certified Solutions Architect and AWS Certified Data Analytics).
• Extensive knowledge of programming or scripting languages (Python Spark etc..
• Expert knowledge of data modelling and strong understanding of various data structures.
• Proven ability to architect highly scalable distributed systems using opensource tools.
• 5 years of experience in data engineering or software engineering.
• 2 years of experience with big data technologies.
• 2 years of experience with ETL processes.
• 2 years of experience working with AWS services (EMR EC2 and S3.
• 5 years of experience with objectoriented design coding and testing patterns.
• Handson experience with Talend for data integration.
• Strong experience with big data batch and streaming tools.
• Solid background in developing commercial or opensource software platforms and largescale data infrastructure.

Apply now!

For more IT jobs please visit www.networkrecruitment.za

If you have not had any response in two weeks please consider the vacancy application unsuccessful. Your profile will be kept on our database for any other suitable roles / positions.

For more information contact :

Senior Specialist Recruitment Consultant

Email :

Required Experience :

Senior IC

Key Skills

Apache Hive,S3,Hadoop,Redshift,Spark,AWS,Apache Pig,NoSQL,Big Data,Data Warehouse,Kafka,Scala

Employment Type : Full-Time

Experience : years

Vacancy : 1

Create a job alert for this search