Data Platform Engineer — Cloud Lakehouse & Pipelines

Blue Pearl HQ; Johannesburg; On-site; ZAR 600,000 - 1,100,000; Full time

Job summary

Blue Pearl HQ is seeking a seasoned Data Engineer to design scalable cloud-native data platforms and lead end-to-end data pipelines. You will work with Python/SQL, lakehouse architectures, and major cloud platforms to modernise ecosystems.

You'll engage clients, support pre-sales with PoCs, mentor colleagues, and drive automation and CI/CD for data workflows in a dynamic consulting environment.

Qualifications

• 3-5 years hands-on data engineering experience.
• Strong Python and/or SQL with query optimisation.
• Experience with relational and non-relational databases.
• Design/build data pipelines and data models.
• Knowledge of lakehouse architectures (medallion pattern).
• Practical cloud platform experience (Azure/AWS/GCP).
• Familiar with Databricks, Snowflake, Delta Lake, PySpark.
• Understanding of dbt, Git, and CI/CD for data workflows.
• Analytical, problem-solving, and stakeholder communication skills.

Responsibilities

• Design and build scalable data platforms using cloud-native and Lakehouse architectures.
• Develop and optimise data pipelines with Python and SQL; utilise ADF, Glue, Dataflow, Databricks, dbt.
• Modernise legacy environments by migrating to cloud-native platforms (Azure, AWS, GCP, Databricks).
• Engage with clients to conceptualise data solutions aligned to business strategy.
• Support sales with pre-sales activities, PoCs, and technical proposals.
• Mentor junior/intermediate consultants; lead technical reviews.
• Identify opportunities to automate, improve data delivery and scalability.
• Collaborate with executives, product, and analytics teams on data infra needs.

Skills

Python

SQL

PySpark

dbt

Education

Bachelor's degree in Computer Science/Information Systems/Information Technology or related field

Master's degree in a relevant field (advantageous)

Microsoft Fabric Data Engineer Associate

Microsoft Azure Data Engineer Associate

Databricks Certified Data Engineer Associate

Google Professional Data Engineer

AWS Certified Data Engineer - Associate

Databricks Certified Data Engineer Professional

Tools

Databricks

Snowflake

Delta Lake

Terraform

Azure Data Factory

AWS Glue

Google Dataflow

BigQuery

Databricks Workflows

Git

CI/CD pipelines

Job description

Blue Pearl HQ is seeking a seasoned Data Engineer to design scalable cloud-native data platforms and lead end-to-end data pipelines. You will work with Python/SQL, lakehouse architectures, and major cloud platforms to modernise ecosystems.

You'll engage clients, support pre-sales with PoCs, mentor colleagues, and drive automation and CI/CD for data workflows in a dynamic consulting environment.