Senior Data Engineer — Big Data & Cloud Pipelines

Tenth Revolution Group; Gauteng; On-site; ZAR 600,000 - 900,000; Full time

Job summary

Tenth Revolution Group in South Africa seeks a Data Engineer to build scalable, secure data pipelines and support analytics. You will work in an agile team, designing infrastructure that handles large data volumes both in batch and real time, leveraging cloud technologies.

Responsibilities include creating data feeds from on-premise to AWS, developing ETL processes with tools like Hadoop/EMR, and delivering data architectures that enable BI insights for business needs.

Qualifications

• Bachelor’s Degree in Computer Science, Computer Engineering, or equivalent.
• AWS Certification.
• Extensive knowledge in different programming or scripting languages.
• 5+ years Data engineering or software engineering experience.
• 2+ years Big Data experience.
• 2+ years’ experience with Extract Transform and Load (ETL) processes.
• 2+ years AWS experience.
• 5 years demonstrated experience with object-oriented design, coding and testing patterns.

Responsibilities

• Architects Data analytics framework.
• Translates complex functional and technical requirements into detailed architecture, design, and high performing software.
• Leads Data and batch/real-time analytical solutions leveraging transformational technologies.
• Drives user story analysis and elaboration, design and development of software applications, testing, and builds automation tools.
• Development and Operations.
• Database Development and Operations.
• Policies, Standards and Procedures.
• Business Continuity & Disaster Recovery.
• Research and Evaluation.
• Creating data feeds from on-premise to AWS Cloud.
• Support data feeds in production on break fix basis.
• Creating data marts using Talend or similar ETL development tool.
• Processing data using the Hadoop paradigm particularly using EMR, AWS’s distribution of Hadoop.
• Develop for Big Data and Business Intelligence including automated testing and deployment.

Skills

Programming languages

Big Data

Data modelling

Distributed systems

ETL

Education

Bachelor’s Degree in Computer Science/Engineering

AWS Certification

Tools

EMR

Hadoop

Talend/ETL tools

Job description

Tenth Revolution Group in South Africa seeks a Data Engineer to build scalable, secure data pipelines and support analytics. You will work in an agile team, designing infrastructure that handles large data volumes both in batch and real time, leveraging cloud technologies.

Responsibilities include creating data feeds from on-premise to AWS, developing ETL processes with tools like Hadoop/EMR, and delivering data architectures that enable BI insights for business needs.