Data Engineer

Level-Up; Johannesburg; On-site; ZAR 450,000 - 600,000; Full time

Job summary

A leading company in data analytics is seeking a skilled Data Engineer to enhance their data capabilities in Johannesburg. You will design and manage data infrastructure, ensuring high availability and accuracy, while collaborating with teams to drive data-driven decision-making. Ideal candidates will possess a degree in a relevant field and have considerable experience in data engineering, including a strong command of cloud technologies and database management.

Qualifications

• 3-5 years in data engineering, analytics, or data management roles.
• 3+ years working with sales, channel, or business development teams.
• Experience in the cybersecurity industry preferred.

Responsibilities

• Build and maintain scalable data pipelines for efficient data ingestion.
• Develop and automate ETL workflows.
• Manage SQL and NoSQL databases effectively.

Skills

Data Pipeline Development

ETL Processes

Database Management

Data Quality Assurance

Collaboration

Performance Optimization

Cloud & Big Data Technologies

Technical Documentation

Education

Bachelors or Honours Degree in Engineering, Data Science, Computer Science, or Information Systems

Tools

AWS

Azure

Google Cloud

Databricks

Power BI

SQL Server

Python

Job description

We are seeking a skilled Data Engineer to design, develop, optimize, and manage robust, highly available data analytics infrastructure, reports, and data models. This role drives the delivery of high-quality data analytics solutions for data ingestion, storage, consumption, and management to generate actionable insights. Reporting to the Technical Team Lead and Software Development Manager, you will play a critical role in advancing our data capabilities.

Key Responsibilities

• Data Pipeline Development: Build and maintain scalable data pipelines for efficient data ingestion, processing, and storage.
• ETL Processes: Develop and automate ETL workflows to integrate data from diverse sources seamlessly.
• Database Management: Manage SQL and NoSQL databases to handle structured and unstructured data effectively.
• Data Quality Assurance: Ensure data integrity, consistency, and security through proactive monitoring.
• Collaboration: Partner with data scientists, analysts, and software engineers to enable data-driven decision-making.
• Documentation: Create clear, detailed documentation for data workflows, architectures, and processes.
• Performance Optimization: Enhance database performance through tuning, query optimization, and indexing.
• Cloud & Big Data Technologies: Support and expand expertise in cloud-based platforms like AWS, Azure, or Google Cloud.

Key Performance Indicators

• Pipeline Efficiency: Percentage of data pipelines executed on schedule.
• Data Accuracy: Reduction in data errors and inconsistencies.
• ETL Performance: Improved speed and efficiency of ETL processes.
• Query Performance: Reduced database query response times.
• Collaboration Success: Number of effective collaborations with data scientists and analysts.
• Code & Documentation Quality: Adherence to coding standards and comprehensive documentation.

Qualifications & Experience

• Bachelors or Honours Degree in Engineering, Data Science, Computer Science, or Information Systems.
• 3 -5 years in data engineering, analytics, or data management roles.
• 3+ years working with sales, channel, or business development teams.
• 3+ years delivering customer-facing projects.
• 3+ years in the cybersecurity industry.
• 3+ years using agile methodologies for project delivery.

Systems & Proficiency

• Certification in AWS, Microsoft, or other Business Intelligence technologies.
• Expertise in Databricks and Python Notebooks.
• Advanced skills in Power BI, DAX, Advanced Excel, Python, SQL Server, and SQL Scripting.
• Proficiency in creating technical architectures, entity relationship diagrams, and process flow diagrams.
• Strong command of Microsoft Office.

Skills & Competencies

• Ability to manage multiple programs, balancing strategic planning with fast-paced execution.
• Strong communication, negotiation, and consensus-building skills with stakeholders and teams.
• Exceptional presentation skills, comfortable presenting to executive leadership.
• Superior analytical, organizational, and time-management abilities.
• Thrives in high-pressure environments and adapts quickly to new skills.
• Trusted to handle confidential information with discretion.
• Self-motivated, team-oriented, with a structured approach and proactive ownership of tasks.