Senior Data Engineer

Cf Consulting; Gauteng; On-site; ZAR 600,000 - 850,000; Full time

Job summary

A leading company seeks a highly skilled Data Engineer to drive technical strategies and implement data solutions. With a focus on data analysis, integration, and modeling, this role requires extensive experience in various data technologies and methodologies. The successful candidate will collaborate across teams to optimize data collection and analysis processes, utilizing cloud environments and big data technologies. Ideal for experienced professionals eager to take on complex data challenges.

Qualifications

• Tertiary degree in Computer Science, B.IT, or related.
• 8-10 years of experience as a Data Engineer / Database Developer.
• Proficiency in SQL and ETL solutions.

Responsibilities

• Identify data sources and automate collection processes.
• Design and build Data Models and Data Warehouses.
• Analyze datasets to discover trends and present data visually.

Skills

Data analysis

Data manipulation

Data modelling

Business analysis

ETL processes

Cloud technologies

Big Data technologies

Education

BSc Computer Science

B.IT

Informatics related degrees

Tools

SQL Server

Oracle

Azure SQL

Databricks

SSIS

Azure Data Factory

Hadoop

Spark

Visual Studio

SQL Server Management Studio

Job description

The Data Engineer will have a knack for data analysis, data manipulation, and data modelling and will be responsible for understanding and driving the overall technical vision and planning of a client's organisation and translating business needs into technical strategy.

Required Qualifications

• Tertiary degree, diploma or certificate in a related field (BSc Computer Science, B.IT or Informatics related degrees).

Experience and Knowledge

• 8-10 years' working experience as a Data Engineer / Database Developer.
• Experience in data mining, large-scale data modelling, and business requirements gathering / analysis.
• Understanding and working experience in data integration and transformation.
• Experience implementing data modelling methodologies like Dimensional Modeling and / or Data Vault.
• Working knowledge of data quality processes and master data management.
• Experience implementing design support systems using Database Management Systems (DBMS) such as SQL Server or Oracle.
• Proficiency in designing and implementing data integration and ETL solutions using SSIS, Azure Data Factory and / or SQL Server stored procedures.
• Understanding of Big Data technologies like Hadoop, MapReduce, Spark, Kafka, Event Hub, and Stream Analytics.
• Experience in database query languages such as T-SQL, ANSI SQL, PL / SQL.
• Some experience developing software solutions using Visual Basic, C++, C#, Java, or Python.
• Experience using SQL Server Management Studio and Visual Studio.
• Experience implementing solutions using Azure SQL databases, Azure Synapse, Azure Storage Accounts, and Databricks.
• Analytical mind and business acumen.
• Additional skills in Tableau, Power BI, mathematics, Scala, Python, or R are advantageous.

Key Responsibilities

• Identify valuable data sources and automate collection processes.
• Preprocess structured and unstructured data.
• Analyze large datasets to discover trends and patterns.
• Design Data Models (Relational and Star Schema).
• Build and develop Data Warehouses.
• Database Administration and Performance Tuning.
• Present data visually and propose solutions to business challenges.
• Collaborate with engineering and product teams.
• Work with Hadoop / Spark frameworks and real-time analytics.
• Architect analytical applications in cloud environments like AWS and Azure.