Senior Data Engineer: Scalable Pipelines & Analytics

Synthesis Software Technologies; Johannesburg; On-site; ZAR 600,000 - 900,000; Full time

Job summary

Synthesis Software Technologies is looking for a Data Engineer to build scalable, secure data pipelines and support an agile team. The role delivers infrastructure that provides insights from raw data, handling large volumes in batch and real-time with cloud and big data tech.

Responsibilities include architecting analytics frameworks, ETL development, data feeds to AWS, and building data marts using Talend, Python, and Spark, while ensuring robust operations and compliance.

Qualifications

• Bachelor’s Degree in Computer Science, Computer Engineering, or equivalent.
• AWS Certification.
• Extensive knowledge in different programming or scripting languages.
• Expert knowledge of data modelling and understanding of different data structures and their benefits and limitations under particular use cases.
• Capability to architect highly scalable distributed systems, using different open source tools.
• 5+ years Data engineering or software engineering experience.
• 2+ years Big Data experience.
• 2+ years’ experience with Extract Transform and Load (ETL) processes.
• 2+ years AWS experience.
• 5 years demonstrated experience with object-oriented design, coding and testing patterns as well as experience in engineering (commercial or open source) software platforms and large-scale data infrastructures.
• Big Data batch and streaming tools.
• Talend.
• AWS: EMR, EC2, S3.
• Python.
• PySpark or Spark

Responsibilities

• Architects Data analytics framework
• Translates complex functional and technical requirements into detailed architecture, design, and high performing software
• Leads Data and batch/real-time analytical solutions leveraging transformational technologies
• Works on multiple projects as a technical lead driving user story analysis and elaboration, design and development of software applications, testing, and builds automation tools
• Development and Operations
• Database Development and Operations
• Policies, Standards and Procedures
• Business Continuity & Disaster Recovery
• Research and Evaluation
• Creating data feeds from on-premise to AWS Cloud
• Support data feeds in production on break fix basis
• Creating data marts using Talend or similar ETL development tool
• Manipulating data using python
• Processing data using the Hadoop paradigm particularly using EMR, AWS’s distribution of Hadoop
• Develop for Big Data and Business Intelligence including automated testing and deployment

Skills

Programming languages

Distributed systems architecture

Data modeling

Python

Big data concepts

Spark / PySpark

ETL concepts

Education

Bachelor’s Degree in Computer Science or Computer Engineering

Tools

AWS

Talend

Spark / PySpark

Job description

Synthesis Software Technologies is looking for a Data Engineer to build scalable, secure data pipelines and support an agile team. The role delivers infrastructure that provides insights from raw data, handling large volumes in batch and real-time with cloud and big data tech.

Responsibilities include architecting analytics frameworks, ETL development, data feeds to AWS, and building data marts using Talend, Python, and Spark, while ensuring robust operations and compliance.