Data Engineer

ADvTECH; Gauteng; On-site; ZAR 200 000 - 300 000; Full time

Job summary

A leading educational group in Gauteng is looking for a skilled Data Engineer responsible for ingesting and transforming data to drive business insights. The ideal candidate will develop visual reports and dashboards, ensuring data accuracy and optimal analytics. Candidates should have a diploma in IT or related fields, along with experience in SQL, BI Development, and Data Engineering. This is an excellent opportunity to engage in data modelling and ETL processes within a dynamic team.

Qualifications

• 5 years as SQL Developer.
• 5 years as BI Developer.
• 3 years in Data Engineering.
• 3 years with Synapse pipelines or Data factory.

Responsibilities

• Ingest and transform data for business insights.
• Develop visual reports and dashboards.
• Collaborate with internal and external teams for data accuracy.
• Monitor and maintain ETL pipelines.

Skills

Data modelling

Excellent communication skills

Analytical and problem solving

Microsoft Power BI

Azure Synapse SQL and pipelines

Microsoft SQL Server Technologies

Education

Diploma in IT or related field

Bachelor's degree in Information Systems, Informatics, Computer Systems or IT

SQL/Data engineering certification

Azure Data certification

Job description

Overall Purpose of the Job (Please provide a summary of why the job was created and what the main responsibilities are)

The data engineer is responsible for ingesting and transforming a wide range of data across the business into formats that can be used by end‑users to develop differentiating business insights. By structuring data requirements into logical data constructs, the Data Engineer enables optimal, physical implementation of the data structure by generating data models. A key deliverable is the development of visual reports, dashboards and KPI scorecards. The role will involve collaborating with internal and external resources to ensure the data analytics team have the required data from the source systems as well as be responsible for the accuracy of data loaded into the Data Warehouse.

Education (Formal Qualification required)

• Diploma in IT or related field
• Bachelor’s degree in Information Systems, Informatics, Computer Systems or Information Technology (NQF 7) or National Diploma/Diploma in Software Development/IT (NQF 6)
• SQL/Data engineering certification
• Azure Data certification

Experience (Minimum Experience required – type and number of years – experience can run concurrently)

• SQL Developer – 5 years
• BI Developer – 5 years
• Data Engineering – 3 years
• Synapse pipelines / Data factory / SSIS – 3 years

Data analysis / data modelling / report building

• Creates data model designs in accordance with recommended best practice
• Collaborates with team members and business to translate business requirements into data requirements
• Analyzes and profiles the source data to understand data quality issues, relationships, patterns, and rules in the data
• Structures data requirements into logical data constructs based on recommended best practice including ERD models and dimensional models to ensure optimal implementation
• Compiles Source to Target Mapping Specifications with the Business Intelligence Business Analyst (BIBA)
• Ensures complete and correct data by identifying definitive or authoritative sources of data; analyzing source data; and identifying gaps
• Enables physical implementation of the data structure by generating the physical data model from the logical data model
• Cleans and maintains databases by restructuring and deleting old data/stored procedures
• Develops visual reports, dashboards and KPI scorecards using Power BI desktop
• Connects to data sources, importing and transforming data for Business Intelligence
• Implements row‑level security on data, demonstrating an understanding of application security layer
• Writes DAX queries in Power BI

ETL and pipeline development

• Assists in designing and implementing scalable and robust processes for ingesting and transforming complex datasets
• Develops the work packages as per the design to deliver elements of the data warehouse architecture (data ingestion, ETL, etc.)
• Provides technical support, troubleshooting, and upgrades setup or support on ETL and data warehouse related issues
• Analyzes and maps data from source systems to target data warehouse
• Participates in all aspects of quality assurance
• Monitors and maintains ETL pipelines and data warehouseDevelops the data dictionary

Task tracking and Support Resolution

• Makes use of DevOps to track tasks, effort estimates, effort completed and repos
• Responds timeously to support calls and tracks issues to completion

Continuous improvement

• Owns the development of elements of centralised, automated data improvement initiatives under the guidance of the Team Lead
• Makes increased contributions by broadening and enhancing individual skills
• Keeps abreast of industry trends and best practice and makes recommendations to management
• Investigates new technologies to improve and optimize standard reporting

Skill

• Data modelling – 15%
• Excellent communication skills – 5%
• Analytical and problem solving – 15%
• Microsoft Power BI – 15%
• Azure Synapse SQL and pipelines – 20%
• Microsoft SQL Server Technologies – 30%

Behavioural Attribute

• Assertive
• Timeous
• Decisiveness
• Team player
• Innovation