Lead Data Engineer

Absa Group; Johannesburg; On-site; ZAR 700,000 - 1,200,000; Full time

Job summary

Absa Group is seeking a senior Data Architect to lead enterprise data retrieval, storage and distribution across a banking data estate. You will guide architecture direction and collaborate with data science teams and stakeholders to translate requirements into robust data solutions.

You will own the end-to-end design and implementation, build scalable data pipelines, CI/CD, and ensure self-service, testability and resilience across platforms, including relational and NoSQL databases.

Qualifications

• Experience leading data architecture across large data estates.
• Ability to translate business data requirements into scalable solutions.
• Strong collaboration with data science teams and stakeholders.

Responsibilities

• Take data retrieval, storage & distribution accountability for a business area.
• Collaborate with data science teams & stakeholders to analyse data requirements and design architecture.
• Lead end-to-end solution design and implementation for data pipelines and storage.
• Build data tools for analytics and data science teams to enable faster insights.
• Create and maintain CI/CD pipelines for deployment to multi-site environments.
• Monitor performance and cost-to-value of data products and services.

Skills

Data architecture

Agile methodologies

CI/CD pipelines

SQL

NoSQL

ETL/ELT design

Data modeling

Cloud platforms

Stakeholder collaboration

Python

Education

Bachelor's degree in Computer Science / Data Engineering

Tools

Jenkins

Kafka

Spark

Hadoop

Airflow

Python

Job description

Empowering Africa’s tomorrow, together…one story at a time. With over 100 years of rich history and strongly positioned as a local bank with regional and international expertise, a career with our family offers the opportunity to be part of this exciting growth journey, to reset our future and shape our destiny as a proudly African group.

Job Summary Leveraging a thorough understanding of the business data requirements & own data subject matter expertise (ahead of trend), set data architecture direction & lead the agile solution design, deployment, implementation & ongoing optimization of enterprise wide data retrieval, storage and distribution across an estate.

Job Description

• Take ‘one stop shop’ accountability for the retrieval, storage, processing & distribution of data across a business area (data eco-system)
• Collaborate with data science teams & broader business & technical stakeholders to analyse customer data requirements & leverage analysis to identify & accurately spec / frame & set the appropriate data architecture for the business area
• Take accountability for deeply understanding the technical landscape and bank wide architecture that is connected to or dependent on the business area supported in order to effectively design & deliver data solutions (architecture, pipeline etc.)
• Leverage expertise in analytical & creative problem solving to synthesise data solution designs (build a solution from its components) beyond the analysis of the problem
• Lead design thinking processes to successfully deliver data solution blueprints & associated roadmaps
• Leverage state of the art relational and No-SQL databases as well integration and streaming platforms do deliver sustainable business specific data solutions
• Lead the end to end solution design & implementation process for all data retrieval, storage & distribution solutions including contributing to and in some cases lead all phases of the development lifecycle including e.g. design process
• Develop high quality data retrieval, storage & distribution design and architecture in a test driven & domain driven / cross domain environment
• Build analytics tools that utilize the data pipeline by quickly producing well-organised, optimised, and documented source code & algorithms to deliver technical data solutions
• Create & Maintain Sophisticated CI / CD Pipelines (authoring & supporting CI/CD pipelines in Jenkins or similar tools and deploy to multi-site environments – supporting and managing your applications all the way to production)
• Automate tasks through appropriate tools and scripting technologies e.g. Ansible, Chef
• Debug existing source code and polish feature sets.
• Assemble large, complex data sets that meet business requirements & manage the data pipeline
• Build infrastructure to automate extremely high volumes of data delivery and
• Creatively solve data volume and scaling challenges.
• Create data tools for analytics and data science teams that assist them in building and optimizing data sets for the benefit of the business
• Ensure designs & solutions support the technical organisation principles of self-service, repeatability, testability, scalability & resilience
• Apply general design patterns and paradigms to deliver technical solutions
• Inform & support the infrastructure build required for optimal extraction, transformation, and loading of data from a wide variety of data sources.
• Continuously optimise, improve & automate data retrieval, storage & distribution processes
• Work across the value chain and in particular with QA & CSO to define and implement quality assurance routines & processes
• Work across the value chain and in particular with QA & CSO to ensure the undisputable separation, security & quality of the organisation’s data
• Meaningfully contribution & ensure solutions align to the design & direction of the Group Architecture & in particular data standards, principles, preferences & practices. Short term deployment must align to strategic long term delivery.
• Meaningfully contribution & ensure solutions align to the design and direction of the Group Infrastructure standards and practices e.g. OLA’s, IAAS, PAAS, SAAS, Containerisation etc.
• Strategically & operationally monitor the performance of data products and services ensuring ongoing optimisation & cost to value for our businesses (think bank wide)
• Stay ahead of the curve on data retrieval, storage & distribution technologies & processes (global best practices & trends) to ensure best practice

People

• Coach & mentor other engineers
• Conduct peer reviews, testing, problem solving within and across the broader team
• Provide technical subject matter expertise and support in the attraction and recruitment of Product Engineers for the organisation
• Participate as a subject matter expert in the development & development planning of the broader dat engineering team
• Support the people change teams in the design of adoption processes (Customer, Employee & 3rd Party Adoption)

Risk & Governance

• Identify technical risks and mitigate these (pre, during & post deployment)
• Update / Design all application documentation aligned to the organization technical standards and risk / governance frameworks
• Create business cases & solution specifications for various governance processes (e.g. CTO approvals)
• Participate in incident management & DR activity – applying critical thinking, problem solving & technical expertise to get to the bottom of major incidents
• Deliver on time & on budget (always)

Education

Absa Bank Limited is an equal opportunity, affirmative action employer. In compliance with the Employment Equity Act 55 of 1998, preference will be given to suitable candidates from designated groups whose appointments will contribute towards achievement of equitable demographic representation of our workforce profile and add to the diversity of the Bank.

Absa Bank Limited reserves the right not to make an appointment to the post as advertised.

At Absa, our story is not a single narrative. It’s thousands of individual journeys unfolding across Africa and beyond. From colleagues serving customers in local communities to teams shaping financial solutions across borders, Absa is built on people who bring their lived experiences, ambitions and perspectives into the workplace every day. Whether you’re starting your career, a seasoned leader or somewhere in between, your contribution is seen, valued and connected to something bigger. Our presence across Africa – which spans personal and business banking, corporate and investment banking, wealth and investment management, and insurance – exists because of the people who show up in their roles with purpose.