Data Engineers

Blue Pearl PTY LTD

Johannesburg

On-site

ZAR 600,000 - 1,200,000

Full time

6 days ago

Be an early applicant

Application generator

Don’t send a generic resume — generate a resume and cover letter tailored to this exact role.

Get past ATS filters

Job summary

Blue Pearl PTY LTD in Johannesburg seeks a Data Engineer to design and build scalable data platforms using cloud-native and lakehouse architectures. You will develop data pipelines with Python, SQL, dbt, and tools like Databricks, ADF, AWS Glue, and BigQuery, while modernising legacy environments and migrating to the cloud.

You will engage with clients to shape data solutions, support pre-sales, and mentor junior engineers, balancing billable work with team growth and knowledge sharing.

Qualifications

3–5 years of hands-on experience in data engineering.
Strong proficiency in Python and/or SQL, including query optimisation.
Experience designing and building data pipelines and data models.
Experience with lakehouse architectures and medallion pattern.
Practical experience with major cloud platforms: Azure, AWS, GCP.
Familiarity with Databricks, Snowflake, Delta Lake, PySpark.
Understanding of dbt for data transformations.
Experience with Git and CI/CD for data workflows.
Strong analytical and problem-solving skills; stakeholder engagement.

Responsibilities

Design and build scalable data platforms using cloud-native and Lakehouse architectures.
Develop and optimise data pipelines using Python, SQL, and tools like Azure Data Factory, AWS Glue, Google Dataflow, Databricks, and dbt.
Modernise legacy data environments by migrating to cloud-native platforms (Azure, AWS, GCP, Fabric, Synapse, BigQuery, Databricks).
Engage with clients to conceptualise data solutions aligned to business strategy.
Support pre-sales activities, proofs-of-concept, and technical proposals.
Mentor junior/intermediate consultants and lead technical reviews.

Skills

Python

SQL

PySpark

dbt

Lakehouse

Git

CI/CD

Data modeling

Cloud platforms

Education

Bachelor's degree in Computer Science / Information Systems / Information Technology

Master's degree advantageous

Tools

Databricks

Snowflake

Delta Lake

Azure Data Factory

AWS Glue

BigQuery

Terraform

Job description

Key Responsibility

You will be assigned a portfolio of client engagements where you will be expected to:

Design and build scalable data platforms using modern cloud-native and Lakehouse architectures
Develop and optimise data pipelines using Python, SQL, and tools such as Azure Data Factory, AWS Glue, Google Cloud Dataflow, Databricks, and dbt
Modernise legacy data environments, migrating from on‑premises solutions to cloud-native platforms such as Microsoft Fabric, Azure Synapse Analytics, AWS Redshift, Google BigQuery, or Databricks
Engage with clients to conceptualise data solutions aligned to their business strategy
Support our sales team with pre‑sales activities, proof‑of‑concept deliveries, and technical proposals
Provide technical guidance and mentorship to junior and intermediate consultants
Lead technical reviews and contribute to consultants' growth plans
Identify opportunities to automate manual processes, optimise data delivery, and improve infrastructure scalability
Work with stakeholders, including executive, product, and analytics teams, to address data infrastructure needs
Drive knowledge sharing through technical blogs, internal forums, and workshops
Balance billable project work with team support responsibilities

Requirements

Data Engineer – Candidate Requirements

Intermediate Level

3–5 years' experience

3–5 years of hands‑on experience in data engineering.
Strong proficiency in Python and/or SQL, including query optimisation.
Experience working with both relational and non‑relational databases.
Experience designing and building data pipelines and data models.
Understanding and practical experience with lakehouse architectures, including the medallion pattern.
Practical experience with at least one major cloud platform, including:
- Microsoft Azure
- AWS
- Google Cloud Platform (GCP)
Familiarity with:
- Databricks
- Snowflake
- Delta Lake
- PySpark
Understanding of data transformation frameworks such as dbt.
Experience with version control using Git.
Understanding of CI/CD practices for data workflows.
Strong analytical and problem‑solving skills.
Ability to perform root‑cause analysis on complex data issues.
Good communication and stakeholder engagement skills.

Senior Level

6–8+ years' experience

6–8+ years of hands‑on experience in data engineering.
All intermediate-level technical requirements, together with demonstrable experience in:
- Leading end‑to‑end data platform delivery.
- Architecting enterprise‑grade lakehouse environments.
- Implementing data mesh patterns.
- Infrastructure‑as‑code using tools such as Terraform, Bicep, AWS CDK or Pulumi.
- DevOps and CI/CD pipelines.
- Working effectively with cross‑functional teams in a dynamic consulting environment.
- Mentoring junior engineers.
- Contributing to technical strategy and solution direction.

Qualifications

Bachelor's degree in:
- Computer Science
- Information Systems
- Information Technology
- or a related field.
Master's degree in a relevant field is advantageous.

Certifications

One or more of the following certifications would be advantageous: