Data Engineer – Banking (CIB)

Blue Pearl HQ; Johannesburg; Hybrid; ZAR 900,000 - 1,500,000; Full time

Job summary

BluePearl Consulting is seeking a strong, experienced Data Engineer to join a software and data engineering team within a Corporate and Investment Banking environment. The role focuses on designing, developing and maintaining scalable data pipelines and cloud-based data platforms, with AWS as the core platform.

The successful candidate will have excellent SQL and Python skills, hands-on ETL/ELT experience, and a proactive, ownership-driven approach to deliver high-quality data solutions with

Qualifications

• Banking/financial services experience preferred.
• Strong hands-on data engineering experience.
• Advanced SQL with optimization, CTEs and window functions.
• Python engineering with reusable modules and testing.
• ETL/ELT, data ingestion and transformation experience.
• Data modelling across relational or analytical models.
• AWS-based cloud data engineering experience.
• Data quality, validation and reconciliation expertise.
• Git, CI/CD, automated testing and production support familiarity.

Responsibilities

• Design, develop and maintain scalable ETL/ELT pipelines and data ingestion processes.
• Write, optimize and troubleshoot advanced SQL queries including CTEs and window functions.
• Develop reusable Python modules and automated tests.
• Build and maintain relational, dimensional and analytical data models.
• Implement data transformations, mappings, validation and reconciliation.
• Develop and maintain data pipelines using AWS data services.
• Implement data orchestration and workflow automation.
• Work with Spark/PySpark as needed for distributed processing.
• Support data warehousing, data lakes and lakehouse architectures.
• Implement data quality rules, logging, monitoring and observability.
• Contribute to CI/CD and automated deployment pipelines.

Skills

SQL

Python

ETL/ELT

Data modelling

Cloud data engineering

Data quality

Git/CI/CD

Testing

Troubleshooting

Tools

AWS

Spark/PySpark

Kafka

Airflow/MWAA

Terraform

Job description

Company: BluePearl Consulting
Number of Vacancies: 1
Industry: Banking / Financial Services
Department: Corporate and Investment Banking (CIB)
Contract Duration: 12 months, with potential extension up to 36 months
Expected Start Date: 1 January 2027
Location: Rosebank, Johannesburg, Gauteng
Work Arrangement: Hybrid - approximately 3 days onsite per week

Job Summary

BluePearl Consulting is seeking a strong, experienced Data Engineer to join a software and data engineering team within a Corporate and Investment Banking (CIB) environment.

The successful candidate will specialise in designing, developing and maintaining scalable data pipelines, data integration solutions and cloud-based data platforms. The role requires strong SQL and Python engineering skills, ETL/ELT expertise, data modelling capabilities and hands‑on cloud data engineering experience, with AWS as the core platform.

The ideal candidate is a proactive self‑starter who takes ownership of deliverables, solves problems independently and maintains high standards of data quality, security and engineering discipline.

Key Responsibilities

• Design, develop and maintain scalable ETL/ELT pipelines and data ingestion processes.

• Write, optimise and troubleshoot advanced SQL queries, including CTEs and window functions.

• Develop reusable Python modules, libraries and automated tests.

• Build and maintain relational, dimensional and analytical data models.

• Implement data transformations, mappings, validation and reconciliation processes.

• Develop and maintain data pipelines using AWS data services.

• Implement data orchestration and workflow automation using appropriate tools.

• Work with distributed processing technologies such as Spark and PySpark where required.

• Support data warehousing, data lakes and lakehouse architectures.

• Implement data quality rules, exception handling and source-to-target reconciliation.

• Apply data security, access control, encryption and privacy practices.

• Implement logging, monitoring, alerting and data observability.

• Contribute to Git-based version control, CI/CD and automated deployment pipelines.

• Develop and maintain unit, integration, regression and reconciliation tests.

• Troubleshoot production incidents, conduct root‑cause analysis and support operational recovery.

• Maintain technical documentation, data mappings, designs and operational runbooks.

• Collaborate with software engineers, data engineers and business stakeholders.

Requirements

Non‑Negotiable Requirements

• Relevant banking or financial services experience.

• Strong hands‑on Data Engineering experience.

• Advanced SQL engineering, including query optimisation, CTEs, window functions and query plans.

• Strong Python engineering, including reusable modules, libraries and testing.

• Practical ETL/ELT, data ingestion and transformation experience.

• Data modelling experience across relational, dimensional or analytical models.

• Cloud data engineering experience, with AWS as the core platform requirement.

• Experience with data quality, validation and reconciliation.

• Knowledge of data security, access control and privacy.

• Familiarity with Git, CI/CD, automated testing and production support.

• Strong troubleshooting, problem-solving and technical documentation skills.

• Ability to work independently and take ownership of deliverables.

• Excellent communication and stakeholder management skills.

• Availability to work onsite in Rosebank approximately 3 days per week.

• Availability to commence on 1 January 2027.

Core Technical Skills

• AWS

• Advanced SQL

• Python

• ETL/ELT

• Data ingestion and transformation

• Data modelling

• Data quality and reconciliation

• Data orchestration

• Git and CI/CD

• Automated testing

• Monitoring and production support

Role‑Critical Technologies

• AWS S3, Glue, Lambda, Athena, Redshift and EMR.

• Spark and PySpark.

• Kafka and event‑driven processing.

• Data warehousing, data lakes and lakehouse engineering.

• Delta Lake and medallion architecture patterns.

• SQS, SNS and EventBridge.

• Airflow/MWAA and Step Functions.

• Terraform and Infrastructure as Code.

• Metadata, data catalogues and lineage.

• Performance optimisation, partitioning and query tuning.

Desirable Capabilities

• Batch processing and workflow scheduling.

• Data governance and data classification.

• Cloud cost optimisation and FinOps awareness.

• Legacy ETL modernisation and cloud migration.

• Data products, interfaces and service‑level agreements/objectives.

• Data mesh and federated governance.

Core Attributes

• Strong banking or financial services awareness.

• Self‑starter who requires minimal supervision.

• Takes ownership of responsibilities and deliverables.

• Proactive and analytical problem‑solver.

• Excellent written and verbal communication.

• Strong attention to detail and data quality.

• Professional stakeholder engagement.

• Smart and professional presentation appropriate to a banking environment.

Recruitment priority

A strong, hands‑on Data Engineer with banking experience, advanced SQL and Python capability, AWS expertise and a proven ability to deliver reliable data solutions independently.