Data Engineer – Banking (CIB)

Blue Pearl PTY; Johannesburg; Híbrido; ZAR 900.000 - 1.300.000; Jornada completa

Descripción de la vacante

BluePearl Consulting is seeking an experienced Data Engineer to join a CIB data engineering team in Rosebank, Johannesburg. The role focuses on designing and maintaining scalable data pipelines, data integration, and cloud-based platforms with AWS as the core platform.

The candidate should excel in SQL and Python, ETL/ELT, data modelling, and data quality. Hybrid work arrangement is expected, with onsite presence ~3 days per week and an anticipated start on 1 January 2027.

Formación

• Strong hands-on Data Engineering experience in a banking/financial services environment.
• Advanced SQL engineering including CTEs and window functions.
• Proficient Python development with reusable modules and tests.
• ETL/ELT, data ingestion and transformation experience.
• Cloud data engineering with AWS as core platform.

Responsabilidades

• Design, develop and maintain scalable ETL/ELT pipelines and ingestion processes.
• Write, optimise and troubleshoot advanced SQL queries (CTEs, window functions).
• Develop reusable Python modules and automated tests.
• Build and maintain relational, dimensional and analytical data models.
• Implement data transformations, mappings, validation and reconciliation.
• Develop data pipelines using AWS data services and orchestration tools.
• Work with Spark/PySpark and distributed processing when required.
• Support data warehousing, data lakes, and lakehouse architectures.
• Implement data quality rules, logging and observability.
• Maintain documentation, runbooks and data mappings.

Conocimientos

Advanced SQL

Python engineering

ETL/ELT

Data modelling

AWS cloud

Data quality

Data governance

CI/CD

Herramientas

Airflow

Terraform

Delta Lake

Spark/PySpark

Kafka

Descripción del empleo

Johannesburg, South Africa | Posted on 10/28/2026

Department: Corporate and Investment Banking (CIB)

Contract Duration: 12 months, with potential extension up to 36 months

Expected Start Date: 1 January 2027

Location: Rosebank, Johannesburg, Gauteng

Work Arrangement: Hybrid - approximately 3 days onsite per week

Job Summary

BluePearl Consulting is seeking a strong, experienced Data Engineer to join a software and data engineering team within a Corporate and Investment Banking (CIB) environment.

The successful candidate will specialise in designing, developing and maintaining scalable data pipelines, data integration solutions and cloud-based data platforms. The role requires strong SQL and Python engineering skills, ETL/ELT expertise, data modelling capabilities and hands-on cloud data engineering experience, with AWS as the core platform.

The ideal candidate is a proactive self-starter who takes ownership of deliverables, solves problems independently and maintains high standards of data quality, security and engineering discipline.

Key Responsibilities

• Design, develop and maintain scalable ETL/ELT pipelines and data ingestion processes.
• Write, optimise and troubleshoot advanced SQL queries, including CTEs and window functions.
• Develop reusable Python modules, libraries and automated tests.
• Build and maintain relational, dimensional and analytical data models.
• Implement data transformations, mappings, validation and reconciliation processes.
• Develop and maintain data pipelines using AWS data services.
• Implement data orchestration and workflow automation using appropriate tools.
• Work with distributed processing technologies such as Spark and PySpark where required.
• Support data warehousing, data lakes and lakehouse architectures.
• Implement data quality rules, exception handling and source-to-target reconciliation.
• Apply data security, access control, encryption and privacy practices.
• Implement logging, monitoring, alerting and data observability.
• Contribute to Git-based version control, CI/CD and automated deployment pipelines.
• Develop and maintain unit, integration, regression and reconciliation tests.
• Troubleshoot production incidents, conduct root-cause analysis and support operational recovery.
• Maintain technical documentation, data mappings, designs and operational runbooks.
• Collaborate with software engineers, data engineers and business stakeholders.

Requirements

Non-Negotiable Requirements

• Relevant banking or financial services experience.
• Strong hands-on Data Engineering experience.
• Advanced SQL engineering, including query optimisation, CTEs, window functions and query plans.
• Strong Python engineering, including reusable modules, libraries and testing.
• Practical ETL/ELT, data ingestion and transformation experience.
• Data modelling experience across relational, dimensional or analytical models.
• Cloud data engineering experience, with AWS as the core platform requirement.
• Experience with data quality, validation and reconciliation.
• Knowledge of data security, access control and privacy.
• Familiarity with Git, CI/CD, automated testing and production support.
• Strong troubleshooting, problem-solving and technical documentation skills.
• Ability to work independently and take ownership of deliverables.
• Excellent communication and stakeholder management skills.
• Availability to work onsite in Rosebank approximately 3 days per week.
• Availability to commence on 1 January 2027.

Core Technical Skills

• AWS
• Advanced SQL
• Python
• ETL/ELT
• Data ingestion and transformation
• Data quality and reconciliation
• Data orchestration
• Git and CI/CD
• Automated testing
• Monitoring and production support

Role-Critical Technologies

• AWS S3, Glue, Lambda, Athena, Redshift and EMR.
• Spark and PySpark.
• Kafka and event-driven processing.
• Data warehousing, data lakes and lakehouse engineering.
• Delta Lake and medallion architecture patterns.
• SQS, SNS and EventBridge.
• Airflow/MWAA and Step Functions.
• Terraform and Infrastructure as Code.
• Metadata, data catalogues and lineage.
• Performance optimisation, partitioning and query tuning.
• Batch processing and workflow scheduling.
• Data governance and data classification.
• Cloud cost optimisation and FinOps awareness.
• Legacy ETL modernisation and cloud migration.
• Data products, interfaces and service-level agreements/objectives.
• Data mesh and federated governance.

Core Attributes

• Strong banking or financial services awareness.
• Self-starter who requires minimal supervision.
• Takes ownership of responsibilities and deliverables.
• Proactive and analytical problem-solver.
• Strong attention to detail and data quality.
• Smart and professional presentation appropriate to a banking environment.

Recruitment priority: A strong, hands-on Data Engineer with banking experience, advanced SQL and Python capability, AWS expertise and a proven ability to deliver reliable data solutions independently.