Associate Data Engineer: CDH

Sanlam Limited; South Africa; On-site; ZAR 500,000 - 800,000; Full time

Job summary

A leading financial services group in South Africa seeks an Associate Data Engineer responsible for developing and managing data infrastructure. Candidates should possess a Bachelor's degree in a quantitative field and significant experience with SQL and data architecture. This role offers opportunities for career growth and development in a collaborative environment focused on diversity and inclusion.

Benefits

Opportunities for growth and development

Inclusive workplace environment

Qualifications

• Relevant experience in a similar environment with data engineering tools.
• Ability to comply with and manage data assets under governance frameworks.
• Experience with data warehousing principles such as Kimball and Vault.

Responsibilities

• Define a structured approach to problem solving.
• Load large, complex data sets and make data available.
• Monitor and fine-tune data vaults and data transformations.

Skills

SQL

Data architecture design

Staging high quality data

Python

Object-oriented programming

Education

Bachelor’s degree in Computer Science or related field

Tools

SAP Hana

Hadoop

ETL tools (e.g., SAP Data Services)

Job description

Main Responsibilities:

• Define a structured approach to problem solving and delivery against it.
• Create role specific design standards, patterns, and principles
• Assist the planning and management of the workload of the team and to ensure delivery
• Load large, complex data sets to and make data available for other data engineers
• Identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, re-designing models for greater scalability
• Working with other data engineers and data modelers, you will design, implement, and manage data vaults, data transformations and the data pipeline
• Identify, design, and implement vault access layers to enable BI products to leverage the data within data vaults
• Monitor and fine-tune data vaults and data transformations on the Cloudera Hadoop stack
• Use modern development and modelling techniques and tools to implement BI and data management solutions, including data quality, metadata and reference data
• Engage with a wide range of technical stakeholders including data scientists, data analysts, business analysts, other data engineers and solutions architects
• Support data stewards to establish and enforce guidelines for data collection, quality improvements, integration, and processes

What will make you successful in this role?

Qualifications:

• Bachelor’s degree in Computer Science, Statistics, Informatics, Information Systems, Engineering or another quantitative field / National Diploma in an Information Technology related discipline preferred

Work Experience:

• The Associate Data Engineer must have relevant experience in a similar environment working with the relevant tools and techniques

Technical Knowledge and Experience:

The Associate Data Engineer is someone with a strong understanding of data, data structures and data sources. Required skills include:

• Application and data engineering background with a solid background in SQL is required
• Knowledge of database management system (DBMS) physical implementation, including tables, joins and SQL querying.
• Data architecture design and delivery experience preferred
• Experience in Database technologies (e.g., SAP Hana, Teradata or similar) or Hadoop components including HDFS, Hive, Spark, Oozie and Impala preferred and highly advantageous.
• Object-oriented/object functional scripting languages (e.g., Python, Java, Scala or related)
• Knowledge and experience of structured data, such as entities, classes, hierarchies, relationships, and metadata.
• Strong Data Engineering background with a specific focus on staging high quality data
• Understanding of data warehousing principles (e.g., Kimball and Vault).
• Ability to comply to and manage data assets under a strict governance framework
• Data warehousing (Kimball and Data Vault patterns are preferred) and dimensional data modelling (e.g., OLAP and MDX experience)
• Experience in developing data pipelines using ETL tools (e.g., SAP Data Services), automation (e.g. Wherescape), scheduling and test automation (e.g. Robot) is desirable
• A solid background in SQL, Information Architecture and ETL procedures is required
• Experience with object-oriented/functional/scripting languages (e.g., Python, Unix Shell scripting, Java, Scala etc.) is preferred but not essential.
• Data Management technologies (e.g., Informatica Data Quality (IDQ), Informatica Enterprise Data Catalog (EDC), Axon, EBX)
• Event/Streaming based data pipelines (e.g., Kafka or Nifi) nice to have

Personal Attributes

Interpersonal savvy - Contributing independently

Decision quality - Contributing independently

Plans and aligns - Contributing independently

Optimises work processes - Contributing independently

Build a successful career with us

We’re all about building strong, lasting relationships with our employees. We know that you have hopes for your future – your career, your personal development and of achieving great things. We pride ourselves in helping our employees to realise their worth.Through its five business clusters – Sanlam Fintech, Sanlam Life and Savings, Sanlam Investment Group, Sanlam Allianz, Santam, as well as MiWay and the Group Office – the group provides many opportunities for growth and development.

Core Competencies

Being resilient - Contributing independently

Collaborates - Contributing independently

Cultivates innovation - Contributing independently

Customer focus - Contributing independently

Drives results - Contributing independently

Turnaround time

The shortlisting process will only start once the application due date has been reached. The time taken to complete this process will depend on how far you progress and the availability of managers.

Our commitment to transformation

The Sanlam Group is committed to achieving transformation and embraces diversity. This commitment is what drives us to achieve a diverse, inclusive and equitable workplace as we believe that these are key components to ensuring a thriving and sustainable business in South Africa. The Group's Employment Equity plan and targets will be considered as part of the selection process.