Senior Data Engineer

Yellowdot Africa; Johannesburg; On-site; ZAR 800,000 - 1,200,000; Full time

Job summary

A leading company in digital solutions is seeking a Senior Data Engineer to design and maintain its data infrastructure. This role involves optimizing data management processes, collaborating with development teams, and ensuring compliance with data governance standards. The ideal candidate will possess a solid background in data engineering, particularly in a fast-paced telecom environment, and will work with cutting-edge technologies.

Qualifications

• 8+ years of data engineering experience, preferably in telecom.
• Bachelor's degree in Computer Science or related field is required.
• Proficient in both SQL and NoSQL databases.

Responsibilities

• Design and maintain databases to support applications and data warehousing.
• Implement and manage ETL pipelines for data processing.
• Collaborate with BI teams to ensure data quality for reporting.

Skills

SQL

NoSQL

ETL

Data Governance

Education

Bachelor's Degree in Computer Science

Tools

Power BI

Tableau

AWS Glue

Job description

Get AI-powered advice on this job and more exclusive features.Our mission at KHM Technology Group is to transform African lives through unique digital solutions, mobile, fintech and adtech products, outstanding service and a passion for the latest innovations.

We offer the best mobile VAS & Games, FinTech solutions, Adtech Solutions and access to Africa's largest Digital Solutions platform.

Discover more at our websites below.Job SummaryThe Data Engineer is responsible for designing, building, and maintaining the group's data infrastructure to support application development, data warehousing, and reporting needs.

The role requires handling data from multiple sources, ensuring it is properly stored, processed, and made accessible to the teams that need it, including developers, analysts, and business stakeholders.The Data Engineer will collaborate with application development teams to integrate databases into software platforms, set up and maintain data pipelines for analytics and BI tools, and ensure data is clean, secure, and readily available for reporting and insights generation.

The ideal candidate will have a strong understanding of both traditional databases and modern data processing frameworks, with the ability to manage data across cloud and on-premises environments.Key Responsibilities1.

Database Design, Management, & Optimization

• Design, develop, and maintain the databases (both SQL and NoSQL) that support the group's applications and platforms.
• Collaborate with developers to define database architecture that supports business logic and application requirements.
• Ensure the databases are optimized for performance, availability, and scalability, handling large volumes of data in a telecom environment.
• Implement data models and structures to support different business needs, from operational data to analytical data.
• Build and maintain efficient, scalable, and reliable ETL pipelines to ingest, clean, and transform data from various internal and external sources into usable formats for reporting, analytics, and BI.
• Automate data pipelines to handle real-time and batch data processing as needed for different use cases (e.g., customer data, billing, traffic analysis).
• Collaborate with the development and operations teams to ensure data flow between systems is seamless, secure, and monitored effectively.3.

Data Warehousing

• Design and implement data warehouses to store large volumes of structured and unstructured data, providing a single source of truth for the group's business and technical data.
• Manage the extraction and aggregation of data into the data warehouse from various systems, including telecom platforms, customer databases, and operational systems.
• Ensure the warehouse is optimized for performance and supports complex queries, reporting, and analytics for business users and analysts.4.

Reporting & Business Intelligence (BI)

• Collaborate with the BI team and business analysts to ensure data is structured and organized in a way that supports efficient reporting and insight generation.
• Support the development of BI dashboards, reports, and analytics solutions by ensuring data availability and integrity in platforms such as Power BI, Tableau, or similar.
• Work closely with business stakeholders to understand reporting needs and develop data structures that support dynamic and accurate reporting.5.

Data Governance, Security, & Compliance

• Implement data governance policies to ensure that data is accurate, secure, and consistent across all systems.
• Ensure proper access controls and security measures are in place to protect sensitive data (e.g., customer data, financial data) from unauthorized access.
• Ensure compliance with telecom industry regulations, data privacy laws (such as GDPR), and any other relevant data protection standards.
• Manage data quality initiatives by identifying and resolving data integrity issues across databases and platforms.6.

Performance Monitoring & Optimization

• Monitor data systems for performance, ensuring that databases, ETL pipelines, and data warehouses run efficiently with minimal downtime.
• Troubleshoot data pipeline issues and implement fixes to ensure the smooth flow of data between systems.
• Continuously identify opportunities for performance improvements, including database tuning, query optimization, and data processing improvements.
• Work closely with developers, DevOps, and application teams to ensure seamless integration of data into applications and platforms.
• Collaborate with analysts, data scientists, and business stakeholders to provide data support for analytics and BI initiatives.
• Stay up to date with the latest trends, technologies, and best practices in data engineering, suggesting improvements and new approaches to data architecture and pipelines.Qualifications and ExperienceRequired :
• Bachelor's Degree in Computer Science, Information Technology, Data Science, or a related field.
• 8+ years of experience in data engineering, database management, or a similar role, preferably in the telecom or VAS industry.
• Proficiency in SQL (e.g., MySQL, PostgreSQL) and NoSQL databases (e.g., MongoDB, Cassandra).
• Hands-on experience with ETL tools (e.g., Apache NiFi, Talend, AWS Glue) and building data pipelines.
• Experience with data warehousing platforms such as Amazon Redshift, Google BigQuery, or Snowflake.
• Familiarity with BI tools like Tableau, Power BI, Looker, or similar platforms.
• Strong understanding of cloud data services, including AWS, Azure, or Google Cloud (e.g., S3, RDS, BigQuery).
• Knowledge of data security best practices and regulations, such as ISO GDPR, POPIA and data encryption.
• Experience with programming languages such as Python, Java, or Scala for data processing and scripting.Preferred :
• Experience in telecom VAS platforms and understanding the data challenges unique to telecom environments (e.g., call data records, SMS, USSD, billing data).
• Experience with big data technologies such as Apache Hadoop, Apache Spark, or Kafka.
• Familiarity with real-time data streaming and processing.
• Knowledge of machine learning pipelines or data processing for AI / ML purposes.
• Data Modelling : Strong skills in data modelling for both relational and non-relational databases, ensuring optimal data structures for operational and analytical needs.
• Problem-Solving : Analytical mindset with the ability to troubleshoot and resolve complex data issues, ensuring the smooth operation of data pipelines and storage systems.
• Collaboration : Ability to work effectively with cross-functional teams, including software developers, BI teams, and business analysts.
• Communication : Excellent communication skills to explain complex data concepts to both technical and non-technical stakeholders.
• Attention to Detail : Strong attention to detail to ensure data accuracy, integrity, and security.
• Continuous Learning : Ability to stay up to date with the latest data engineering tools, technologies, and best practices to drive continuous improvement.Key Metrics for Success
• Reliable, scalable, and optimized data pipelines that support the group's VAS applications and business intelligence needs.
• High data availability, performance, and accuracy across all platforms, minimizing downtime and data issues.
• Positive feedback from business stakeholders on data availability, quality, and the efficiency of reporting and BI tools.
• Effective implementation of security and compliance measures for data management, ensuring data protection and legal compliance.
• Continuous optimization of database and data processing performance, enabling the group to scale operations as needed.Why Join Us?
• Be a critical part of building the data infrastructure for a growing digital solutions group, contributing to impactful applications and business insights.
• Work with cutting-edge technologies in data engineering, from cloud platforms to modern data processing frameworks.
• Collaborate with talented teams to deliver data-driven solutions that enhance business performance and customer experience.Grow professionally in a dynamic, fast-paced, and technology-driven environment, with opportunities for career advancement.This role is critical to KHM Technology's mission of delivering innovative and scalable solutions across Africa.

If you are a seasonedSenior Data Engineerpassionate about data and systems, cloud, code, and collaboration, we invite you to apply!Email Contact :

• levelSeniority levelMid-Senior levelEmployment typeEmployment typeFull-timeJob functionJob functionEngineering and Information TechnologyIndustriesMedia ProductionReferrals increase your chances of interviewing at YellowDot Africa by 2xGet notified about new Senior Software Architect jobs in Johannesburg Metropolitan Area.City of Johannesburg, Gauteng, South Africa 3 days agoJohannesburg Metropolitan Area 4 days agoJohannesburg, Gauteng, South Africa 11 hours agoJohannesburg, Gauteng, South Africa 2 days agoCity of Johannesburg, Gauteng, South Africa 2 weeks agoJohannesburg, Gauteng, South Africa 1 week agoJohannesburg, Gauteng, South Africa 2 days agoCity of Johannesburg, Gauteng, South Africa 2 days agoCity of Johannesburg, Gauteng, South Africa 3 weeks agoJohannesburg, Gauteng, South Africa 1 month agoWoodmead, Gauteng, South Africa 1 month agoWoodmead, Gauteng, South Africa 1 month agoJohannesburg, Gauteng, South Africa 1 week agoLead Software Engineer (OCR / IDP) | 6-Month Contract | Johannesburg or Cape TownJohannesburg, Gauteng, South Africa 1 week agoWoodmead, Gauteng, South Africa 1 month agoJohannesburg Metropolitan Area 3 days agoSenior Software Quality Assurance EngineerCity of Johannesburg, Gauteng, South Africa 5 days agoSenior C# .NET Developer (Financial Markets / Risk Platform) Johannesburg, Gauteng, South Africa 2 days agoSenior C# Developer, Trilogy (Remote) - $60, / year USDJohannesburg, Gauteng, South Africa $30.00-$30.00 2 days agoLead Software Engineer - OCR / IDP (AWS Environment)Johannesburg Metropolitan Area 3 days agoLead Software Engineer, Back End – L5 (Bangkok based – Relocation provided)City of Johannesburg, Gauteng, South Africa 2 months agoWe're unlocking community knowledge in a new way.

Experts add insights directly into each article, started with the help of AI.

J Ljbffr

Create a job alert for this search

Senior Data Engineer • Johannesburg, Gauteng