Specialist Data Engineer

Absa Bank; Gauteng; On-site; ZAR 500 000 - 700 000; Full time

Job summary

A leading financial institution located in South Africa is seeking a Data Engineer to design and maintain data retrieval, processing, storage, and distribution solutions. The ideal candidate should have over 3 years of relevant experience, proficiency in Hadoop, and knowledge of Spark or AWS. Responsibilities include building analytics tools, managing CI/CD pipelines, and ensuring data quality. This position offers opportunities for professional growth and contributes to innovative financial solutions.

Qualifications

• 3+ years relevant experience.
• Experienced in lake formation.

Responsibilities

• Design data retrieval, storage & distribution solutions.
• Build analytics tools that utilize the data pipeline.
• Create and maintain sophisticated CI/CD pipelines.
• Automate tasks through appropriate tools and scripting technologies.
• Monitor the performance of data solutions designs.

Skills

Proficiency in Hadoop

Spark knowledge

Ability to adapt to in-house built ETL tools

Education

Bachelor's Degree in Information Technology

Tools

AWS

Ansible

Jenkins

Job description

Empowering Africa's tomorrow, together…one story at a time.

With over years of rich history and strongly positioned as a local bank with regional and international expertise, a career with our family offers the opportunity to be part of this exciting growth journey, to reset our future and shape our destiny as a proudly African group.

Job Summary

Work embedded as a member of squad OR; across multiple squads to produce, test, document and review algorithms & data specific source code that supports the deployment & optimisation of data retrieval, processing, storage and distribution for a business area.

Job Description

Data Architecture & Data Engineering

• Design data retrieval, storage & distribution solutions (and OR components thereof) including contributing to all phases of the development lifecycle e.g. design process*
• Develop high quality data processing, retrieval, storage & distribution design in a test driven & domain driven / cross domain environment*
• Build analytics tools that utilize the data pipeline by quickly producing well-organised, optimized, and documented source code & algorithms to deliver technical data solutions*
• Create & Maintain Sophisticated CI / CD Pipelines (authoring & supporting CI / CD pipelines in Jenkins or similar tools and deploy to multi-site environments – supporting and managing your applications all the way to production)*
• Automate tasks through appropriate tools and scripting technologies e.g. Ansible, Chef*
• Debug existing source code and polish feature sets.
• Assemble large, complex data sets that meet business requirements & manage the data pipeline*
• Build infrastructure to automate extremely high volumes of data delivery*
• Create data tools for analytics and data science teams that assist them in building and optimizing data sets for the benefit of the business*
• Ensure designs & solutions support the technical organisation principles of self-service, repeatability, testability, scalability & resilience*
• Apply general design patterns and paradigms to deliver technical solutions*
• Inform & support the infrastructure build required for optimal extraction, transformation, and loading of data from a wide variety of data sources*
• Support the continuous optimisation, improvement & automation of data processing, retrieval, storage & distribution processes*
• Ensure the quality assurance and testing of all data solutions aligned to the QA Engineering & broader architectural guidelines and standards of the organisation*
• Implement & align to the Group Security standards and practices to ensure the undisputable separation, security & quality of the organisation's data*
• Meaningfully contribute to & ensure solutions align to the design & direction of the Group Architecture & in particular data standards, principles, preferences & practices.
• Short term deployment must align to strategic long term delivery.
• Meaningfully contribute to & ensure solutions align to the design and direction of the Group Infrastructure standards and practices e.g. OLA's, IAAS, PAAS, SAAS, Containerisation etc.
• Monitor the performance of data solutions designs & ensure ongoing optimization of data solutions
• Stay ahead of the curve on data processing, retrieval, storage & distribution technologies & processes (global best practices & trends) to ensure best practice
• People Coach & mentor other engineers
• Conduct peer reviews, testing, problem solving within and across the broader team
• Build data science team capability in the use of data solutions
• Risk & Governance: Identify technical risks and mitigate these (pre, during & post deployment)*
• Update / Design all application documentation aligned to the organization technical standards and risk / governance frameworks*
• Create business cases & solution specifications for various governance processes (e.g. CTO approvals)*
• Participate in incident management & DR activity – applying critical thinking, problem solving & technical expertise to get to the bottom of major incidents*
• Deliver on time & on budget (always)

Qualifications

• 3+ years relevant experience*
• Proficiency in Hadoop required
• Spark and / or AWS knowledge a distinct advantage
• Experienced in lake formation
• Ability to adapt to inhouse built ETL tools
• Education: Bachelor's Degree : Information Technology

Absa Bank Limited is an equal opportunity, affirmative action employer.

In compliance with the Employment Equity Act 55 of , preference will be given to suitable candidates from designated groups whose appointments will contribute towards achievement of equitable demographic representation of our workforce profile and add to the diversity of the Bank.

Absa Bank Limited reserves the right not to make an appointment to the post as advertised*