Data Engineer Centurion

Mediro ICT Recruitment

Centurion

On-site

ZAR 600,000 - 900,000

Full time

14 days+

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Job summary

Mediro ICT Recruitment is seeking a data engineer to design, implement, and deploy data transformations using Python, PySpark, and SQL in Azure Databricks and Azure SQL. The role focuses on building robust ETL pipelines, data lake curation, and providing data for analytics across multiple Azure services.

You will work with streaming data, BI concepts, and containerised CI/CD pipelines in a dynamic environment with evolving requirements, ensuring high attention to detail.

Qualifications

  • Bachelor's degree in a STEM field; solid foundation in data engineering concepts.
  • Experience designing, developing and deploying ETL/ELT pipelines.

Responsibilities

  • Develop ETL pipelines using Python and SQL in Azure Databricks and Azure SQL.
  • Combine and curate data in a central data lake for analytics.
  • Serve data for applications and analytics via SQL, CosmosDB, Synapse and related tech.
  • Build transformation pipelines into dimensions and facts using BI concepts.
  • Implement streaming pipelines with IoT Hub, Event Hub, Databricks streaming and Azure stream tech.

Skills

Python
PySpark
SQL
Solution Architecture
API Design
Containers
CI/CD
Azure Cloud
Data stream patterns and technology
Data engineering design patterns

Education

Bachelor's Degree in Computer Science, Software Engineering, or Engineering

Tools

Azure Databricks
Azure SQL
ARM templates
CosmosDB
Server Synapse

Job description

Job Description

Profile Outline
A technology focused individual, data engineers will need to design, configure, develop, and deploy data transformations.

Key Responsibilities

  1. Develop ETL pipelines. The data transformations will be developed in Azure Databricks using Python and on Azure SQL using T-SQL and deployed using ARM templates.
  2. Combine and curate data in a central data lake.
  3. Serve data for application and analytics through a variety of technologies such as SQL, Server Synapse, CosmosDB, and TSI.
  4. Build transformation pipelines into dimensions and facts; therefore, a strong knowledge of standard BI concepts is mandatory.
  5. Build stream pipelines leveraging IoT Hub, Event Hub, Databricks streaming, and other Azure stream technologies.
  6. Work in a fluid environment with changing requirements whilst maintaining absolute attention to detail.

Knowledge
Bachelor's Degree in Computer Science, Software Engineering, or Engineering.

Key Skills
Python
PySpark
SQL
Solution Architecture
API Design
Containers
CI/CD
Azure Cloud
Data stream patterns and technology
Data engineering design patterns

Package & Remuneration

Salary Market Related

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Senior Data Engineer
Senior Data Engineer

Sabenza IT & Recruitment • Centurion

On-site
ZAR 900,000 - 1,400,000
Senior Data Engineer — Azure Data Platform & Analytics
Senior Data Engineer — Azure Data Platform & Analytics

Cf Consulting • Gauteng

On-site
ZAR 700,000 - 1,000,000
Competitive salary
On-site work environment
Medical, dental, and vision insurance
+2
Azure Data Engineer
Azure Data Engineer

Blue Pearl PTY • Johannesburg

Hybrid
ZAR 600,000 - 900,000
Data Engineer
Data Engineer

Parvana • Kraaifontein

Hybrid
ZAR 650,000 - 900,000
Data Engineer
Data Engineer

Network Contracting • Durban

On-site
ZAR 600,000 - 1,000,000
Data Engineer
Data Engineer

Parvana • Bellville

Hybrid
ZAR 700,000 - 1,100,000
Senior Data Engineer
Senior Data Engineer

Sabenza It & Recruitment • Gauteng

On-site
ZAR 600,000 - 900,000
Data Engineer
Data Engineer

Network Contracting • Johannesburg

On-site
ZAR 600,000 - 1,200,000
Data Engineer (Azure Data Factory & Databricks) - Johannesburg - up to R500 per hour
Data Engineer (Azure Data Factory & Databricks) - Johannesburg - up to R500 per hour

e-Merge IT Recruitment • Johannesburg

On-site
ZAR 413,000 - 689,000
Data Engineer
Data Engineer

AES Global (South Africa) • Cape Town

Hybrid
ZAR 600,000 - 900,000