Data Engineer – Banking (Cib)

Blue Pearl Pty Ltd; Johannesburg; Hybrid; ZAR 950,000 - 1,350,000; Full time

Benefits offered by this job

Hybrid work arrangement

Job summary

BluePearl Consulting is seeking a Data Engineer to join a corporate and investment banking data team in Rosebank. The role focuses on scalable data pipelines, cloud-based platforms on AWS and strong SQL/Python engineering, with hybrid onsite work and a Jan 1 2027 start.

You will design ETL/ELT processes, model data, and implement data quality, security and observability across data lakes, warehouses and lakehouse architectures.

Qualifications

• Relevant banking or financial services experience.
• Hands-on Data Engineering experience.
• Advanced SQL engineering including CTEs and window functions.
• Strong Python engineering with testing.
• ETL/ELT, data ingestion and transformation experience.
• Data modelling across relational, dimensional or analytical models.
• Cloud data engineering with AWS as core platform.
• Data quality, validation and reconciliation knowledge.
• Data security and privacy awareness.
• Familiarity with Git, CI/CD and production support.
• Ability to work independently and communicate clearly.
• Availability to work onsite in Rosebank ~3 days per week.
• Availability to commence on 1 January 2027.

Responsibilities

• Design, develop and maintain scalable ETL/ELT pipelines and ingestion.
• Write and optimise advanced SQL queries with CTEs and window functions.
• Develop reusable Python modules and automated tests.
• Build and maintain relational, dimensional and analytical models.
• Implement data transformations, mappings and validation processes.
• Develop data pipelines using AWS data services.
• Orchestrate data workflows with appropriate tools.
• Work with Spark/PySpark where required.
• Support data warehouses, data lakes and lakehouses.
• Implement data quality rules, reconciliation and observability.
• Ensure data security, access controls and privacy practices.
• Contribute to CI/CD, version control and deployment pipelines.
• Develop unit/integration/regression tests; troubleshoot incidents.
• Document data mappings, designs and runbooks; collaborate with teams.

Skills

Advanced SQL

Python

ETL/ELT

Data ingestion

Data modelling

AWS

Spark

Git/CI/CD

Data quality

Data governance

Airflow

Delta Lake

Kafka

Tools

Kafka

Airflow

Terraform

MWAA

Delta Lake

Job description

Department: Corporate and Investment Banking (CIB)

Contract Duration: 12 months, with potential extension up to 36 months

Expected Start Date: 1 January 2027

Location: Rosebank, Johannesburg, Gauteng

Work Arrangement: Hybrid – approximately 3 days onsite per week

Job Summary

BluePearl Consulting is seeking a strong, experienced Data Engineer to join a software and data engineering team within a Corporate and Investment Banking (CIB) environment.

The successful candidate will specialise in designing, developing and maintaining scalable data pipelines, data integration solutions and cloud-based data platforms. The role requires strong SQL and Python engineering skills, ETL/ELT expertise, data modelling capabilities and hands-on cloud data engineering experience, with AWS as the core platform.

The ideal candidate is a proactive self-starter who takes ownership of deliverables, solves problems independently and maintains high standards of data quality, security and engineering discipline.

Key Responsibilities

Design, develop and maintain scalable ETL/ELT pipelines and data ingestion processes.

Write, optimise and troubleshoot advanced SQL queries, including CTEs and window functions.

Develop reusable Python modules, libraries and automated tests.

Build and maintain relational, dimensional and analytical data models.

Implement data transformations, mappings, validation and reconciliation processes.

Develop and maintain data pipelines using AWS data services.

Implement data orchestration and workflow automation using appropriate tools.

Work with distributed processing technologies such as Spark and PySpark where required.

Support data warehousing, data lakes and lakehouse architectures.

Implement data quality rules, exception handling and source-to-target reconciliation.

Apply data security, access control, encryption and privacy practices.

Implement logging, monitoring, alerting and data observability.

Contribute to Git-based version control, CI/CD and automated deployment pipelines.

Develop and maintain unit, integration, regression and reconciliation tests.

Troubleshoot production incidents, conduct root-cause analysis and support operational recovery.

Maintain technical documentation, data mappings, designs and operational runbooks.

Collaborate with software engineers, data engineers and business stakeholders.

Requirements

Non-Negotiable Requirements

Relevant banking or financial services experience.

Strong hands-on Data Engineering experience.

Advanced SQL engineering, including query optimisation, CTEs, window functions and query plans.

Strong Python engineering, including reusable modules, libraries and testing.

Practical ETL/ELT, data ingestion and transformation experience.

Data modelling experience across relational, dimensional or analytical models.

Cloud data engineering experience, with AWS as the core platform requirement.

Experience with data quality, validation and reconciliation.

Knowledge of data security, access control and privacy.

Familiarity with Git, CI/CD, automated testing and production support.

Strong troubleshooting, problem-solving and technical documentation skills.

Ability to work independently and take ownership of deliverables.

Excellent communication and stakeholder management skills.

Availability to work onsite in Rosebank approximately 3 days per week.

Availability to commence on 1 January 2027.

Core Technical Skills

AWS

Advanced SQL

Python

ETL/ELT

Data ingestion and transformation

Data quality and reconciliation

Data orchestration

Git and CI/CD

Automated testing

Monitoring and production support

Role-Critical Technologies

AWS S3, Glue, Lambda, Athena, Redshift and EMR.

Spark and PySpark.

Kafka and event-driven processing.

Data warehousing, data lakes and lakehouse engineering.

Delta Lake and medallion architecture patterns.

SQS, SNS and EventBridge.

Airflow/MWAA and Step Functions.

Terraform and Infrastructure as Code.

Metadata, data catalogues and lineage.

Performance optimisation, partitioning and query tuning.

Batch processing and workflow scheduling.

Data governance and data classification.

Cloud cost optimisation and FinOps awareness.

Legacy ETL modernisation and cloud migration.

Data products, interfaces and service-level agreements/objectives.

Data mesh and federated governance.

Core Attributes

Strong banking or financial services awareness.

Self-starter who requires minimal supervision.

Takes ownership of responsibilities and deliverables.

Proactive and analytical problem-solver.

Strong attention to detail and data quality.

Smart and professional presentation appropriate to a banking environment.

Recruitment priority: A strong, hands-on Data Engineer with banking experience, advanced SQL and Python capability, AWS expertise and a proven ability to deliver reliable data solutions independently.