Data Engineer (KG)

Mediro ICT Recruitment

Centurion

On-site

ZAR 700,000 - 900,000

Full time

14 days+

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Job summary

Mediro ICT Recruitment seeks a data engineering specialist to design, develop and deploy data transformations. You will work on ETL pipelines in Azure Databricks using Python and on Azure SQL with T-SQL, deploying via ARM templates.

You will consolidate data into a central data lake and serve analytics-ready data through SQL, Synapse, CosmosDB and related tech, building pipelines into dimensions and facts with strong BI knowledge. IoT streaming experience is a plus.

Qualifications

  • Bachelor degree in Computer Science, Software Engineering or Engineering.

Responsibilities

  • Develop ETL pipelines using Python in Azure Databricks and T-SQL in Azure SQL, deployed with ARM templates.
  • Combine and curate data in a central data lake.
  • Serve data for applications and analytics through SQL, Azure Synapse, CosmosDB and related technologies.
  • Build transformations into dimensions and facts with BI concepts.
  • Create stream pipelines with IoT Hub, Event Hub and Databricks streaming.
  • Adapt to changing requirements while maintaining high attention to detail.

Skills

Python
PySpark
SQL
Solution Architecture
API Design
Containers
CI/CD
Azure Cloud
Data stream patterns and technology
Data engineering design patterns

Education

Bachelor Degree in Computer Science, Software Engineering or Engineering

Tools

Azure Databricks
Azure SQL
ARM templates
CosmosDB
Azure Synapse

Job description

Job Description

Profile Outline
A technology focussed individual, data engineers will need to design, configure, develop and deploy data transformations.

Key Responsibilities

  1. Develop ETL pipelines. The data transformations will be developed in Azure Databricks using Python and on Azure SQL using T-SQL and deployed using ARM templates.
  2. Combine and curate data in a central data lake.
  3. Serve data for application and analytics through a variety of technologies such as SQL, Server Synapse, CosmosDB and TSI.
  4. Build transformation pipelines into dimensions and facts; a strong knowledge of standard BI concepts is mandatory.
  5. Build stream pipelines leveraging IoT Hub, Event Hub, Databricks streaming and other Azure stream technologies.
  6. Work in a fluid environment with changing requirements whilst maintaining absolute attention to detail.

Knowledge
Bachelor Degree in Computer Science, Software Engineering or Engineering.

Key Skills
Python
PySpark
SQL
Solution Architecture
API Design
Containers
CI/CD
Azure Cloud
Data stream patterns and technology
Data engineering design patterns

Package & Remuneration

Salary Market Related

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Data Engineer
Data Engineer

Network Contracting • Durban

On-site
ZAR 600,000 - 1,000,000
Data Engineer
Data Engineer

Indsafri • South Africa

On-site
ZAR 600,000 - 900,000
Azure Data Engineer
Azure Data Engineer

Blue Pearl PTY • Johannesburg

Hybrid
ZAR 600,000 - 900,000
Senior Data Engineer - Cloud Data Architect & Analytics
Senior Data Engineer - Cloud Data Architect & Analytics

Boardroom Appointments • Johannesburg

On-site
ZAR 800,000 - 1,200,000
Data Engineer
Data Engineer

Boardroom Appointments • Johannesburg

On-site
ZAR 500,000 - 900,000
Data Engineer
Data Engineer

Tumaini Consulting • Cape Town

On-site
ZAR 450,000 - 600,000
Data Engineer
Data Engineer

Rifumo Empowerment Holdings • Gauteng

On-site
ZAR 334,800 - 558,000
Data Engineer
Data Engineer

PBT Group • Johannesburg

On-site
ZAR 600,000 - 1,200,000
Data Engineer
Data Engineer

Network Contracting • Johannesburg

On-site
ZAR 600,000 - 1,200,000
Data Engineer
Data Engineer

Inkhanyeti Solutions • Johannesburg

On-site
ZAR 600,000 - 750,000