Cloud Data Platform Engineer: Lakehouse & Pipelines Lead

Blue Pearl; Johannesburg; On-site; ZAR 900,000 - 1,500,000; Full time

Job summary

Blue Pearl in Johannesburg seeks an experienced Data Engineer to design scalable data platforms and modernise cloud-native data ecosystems for enterprise clients. You will lead end-to-end data pipelines using Python/SQL and tools like Azure Data Factory, AWS Glue, Google Dataflow, Databricks and dbt.

The role requires strong cloud experience across Azure, AWS or GCP, with a focus on CI/CD, data governance and stakeholder collaboration to deliver robust analytics and insights.

Qualifications

• 3–5 years hands-on data engineering experience for intermediate level.
• Strong Python/SQL skills with query optimisation.
• Experience with relational and non-relational databases; data pipelines and models.

Responsibilities

• Design and build scalable data platforms using cloud-native and lakehouse architectures.
• Develop data pipelines with Python, SQL and tools like ADF, Glue, Dataflow, Databricks and dbt.
• Modernise on-prem environments to cloud-native platforms (Fabric, Synapse, Redshift, BigQuery, Databricks).
• Engage with clients to conceptualise data solutions aligned to business strategy.
• Support pre-sales activities, PoCs, and technical proposals.
• Provide tech guidance and mentorship to junior/intermediate consultants.
• Lead technical reviews and contribute to growth plans.
• Identify opportunities to automate processes and improve scalability.
• Collaborate with executives, product and analytics teams on data needs.
• Drive knowledge sharing via blogs, internal forums and workshops.
• Balance billable work with team support responsibilities.

Skills

Python

PySpark

SQL

dbt

Git

Education

Bachelor's degree in Computer Science / Information Systems / Information Technology or related field

Master's degree is advantageous

Tools

Azure Data Factory

AWS Glue

Google Dataflow

Databricks

dbt

Job description

Blue Pearl in Johannesburg seeks an experienced Data Engineer to design scalable data platforms and modernise cloud-native data ecosystems for enterprise clients. You will lead end-to-end data pipelines using Python/SQL and tools like Azure Data Factory, AWS Glue, Google Dataflow, Databricks and dbt.

The role requires strong cloud experience across Azure, AWS or GCP, with a focus on CI/CD, data governance and stakeholder collaboration to deliver robust analytics and insights.