Job title : Senior Specialist Cloud Data Engineer
Job Location : Western Cape, Cape Town Deadline : August 08, 2025 Quick Recommended Links
- Jobs by Location
- Job by industries
StartFragment
Role Purpose / Business Unit :
The primary purpose of this, Senior Data Engineer role, is to design, develop, and maintain robust, scalable, and secure data and analytics infrastructure that supports batch and real-time data processing at scale.This includes managing on-premises big data platforms, edge applications, and cloud deployments to ensure seamless integration and optimal performance across all environments.The role is crucial in driving innovation, ensuring data integrity, and delivering actionable insights that empower the organization to make data-driven decisions.Key Objectives
Data Pipeline Excellence : Build and maintain efficient data pipelines that handle large volumes of data with high reliability and performance.Edge and Cloud Integration : Seamlessly integrate edge applications and cloud services to provide real-time data access and machine learning capabilities.Innovation and Improvement : Continuously seek opportunities to enhance the data infrastructure, adopt new technologies, and improve processes.Collaboration and Leadership : Work closely with cross-functional teams to understand their data needs and provide technical leadership and guidance.Your responsibilities will include :
Data Pipeline Management : Design, develop, and maintain scalable data pipelines using batch technologies (like Spark, NiFi, and Hive) and real-time technologies (like Kafka, Flink, Spark streams).Edge Application Development : Implement and manage edge applications using MongoDB and CassandraDB, ensuring efficient data processing and storage.Microservices and Containerization : Develop and deploy microservices in an OpenShift containerized environment, utilizing tools like Nginx API Gateway for real-time data access.Cloud Deployment and Management : Implement and support similar use cases in AWS, ensuring seamless integration between on-premise and cloud environments.Performance Monitoring and Optimization : Continuously monitor and optimize the performance of data pipelines, applications, and services.Security and Compliance : Ensure all systems and data processes comply with relevant security standards and regulations.Technology Stack Selection : Recommend / make decisions on the appropriate technologies and tools to use for various components of the data and analytics infrastructure.Architecture Design : Define the architecture for data pipelines, edge applications, and microservices to ensure scalability and reliability.Resource Allocation : Allocate resources effectively to balance performance, cost, and scalability across on-premise and cloud environments.Data Governance and Compliance : Establish and enforce data governance policies to ensure data quality, security, and compliance.Incident Management : Lead the response to any incidents or outages, ensuring quick resolution and minimal impact on operations.Innovation and Improvement : Continuously seek opportunities to improve processes, adopt new technologies, and drive innovation within the team.The ideal candidate for this role will have :
3 year Computer Science, IT or IS degree or diploma or related field is essential5 to 8 years relevant experienceRelevant AWS, GCP or Azure cloud certification at professional or associate levelData engineering or related software development experienceAgile exposure working with Kanban or ScrumKey Competencies
Technical Proficiency : Strong skills in programming languages such as Python, Java, or ScalaBig Data Technologies : Expertise in tools like Spark, Hive, parquet, iceberg, etc.Database Management : Proficiency with both relational & NoSQL databases (e.g., MongoDB, CassandraDB)Cloud Computing : Experience with cloud platforms like AWS / GCP, including services for data storage and processing.Containerization & Microservices : Knowledge of containerization technologies (e.g., Docker, Kubernetes) and microservices architecture, particularly in OpenShift, AWS ECS and GCP GKE environmentsAPI Management : Experience with API gateways like Nginx and developing APIs for real-time data accessKnowledge Areas
Distributed Systems : In-depth understanding of distributed computing principles and technologiesData Engineering : Knowledge of data pipeline design, ETL processes, and data integrationSecurity and Compliance : Familiarity with data security practices and regulatory compliance requirementsPerformance Optimization : Techniques for monitoring and optimizing the performance of data systems and applicationsEdge Computing : Understanding of edge computing concepts and technologies for processing data closer to the sourceExperience
Hands-On Experience : Several years of experience in software engineering, data engineering, or cloud engineering rolesProject Management : Experience managing complex projects, preferably in a big data or cloud environment.Team Collaboration : Proven ability to work effectively in cross-functional teams and communicate technical concepts to non-technical stakeholdersProblem-Solving : Strong analytical and problem-solving skills, with a track record of addressing complex technical challengesContinuous Learning : Commitment to staying updated with the latest technologies and best practices in the fieldWe make an impact by offering :
Enticing incentive programs and competitive benefit packagesRetirement funds, risk benefits, and medical aid benefitsCell phone and data benefits, advantages fibre connection discounts, and exclusive staff discounts offered in collaboration with partner companiesClosing date for Applications : 04 August 2025.
EndFragment
Research / Data Analysis jobs