Data Engineer & Architect: Scalable Data Pipelines

Absa Bank; Gauteng; On-site; ZAR 500 000 - 700 000; Full time

Job summary

A leading financial institution located in South Africa is seeking a Data Engineer to design and maintain data retrieval, processing, storage, and distribution solutions. The ideal candidate should have over 3 years of relevant experience, proficiency in Hadoop, and knowledge of Spark or AWS. Responsibilities include building analytics tools, managing CI/CD pipelines, and ensuring data quality. This position offers opportunities for professional growth and contributes to innovative financial solutions.

Qualifications

• 3+ years relevant experience.
• Experienced in lake formation.

Responsibilities

• Design data retrieval, storage & distribution solutions.
• Build analytics tools that utilize the data pipeline.
• Create and maintain sophisticated CI/CD pipelines.
• Automate tasks through appropriate tools and scripting technologies.
• Monitor the performance of data solutions designs.

Skills

Proficiency in Hadoop

Spark knowledge

Ability to adapt to in-house built ETL tools

Education

Bachelor's Degree in Information Technology

Tools

AWS

Ansible

Jenkins

Job description

A leading financial institution located in South Africa is seeking a Data Engineer to design and maintain data retrieval, processing, storage, and distribution solutions. The ideal candidate should have over 3 years of relevant experience, proficiency in Hadoop, and knowledge of Spark or AWS. Responsibilities include building analytics tools, managing CI/CD pipelines, and ensuring data quality. This position offers opportunities for professional growth and contributes to innovative financial solutions.