Senior Data Engineer

iDbase Software; Johannesburg; On-site; ZAR 200 000 - 300 000; Full time

Job summary

A leading software company in South Africa is seeking a Senior Data Engineer to design and maintain scalable data solutions. The ideal candidate will have over 6 years of experience in data engineering, strong SQL skills, and proficiency with cloud platforms like AWS or Azure. This role offers a dynamic work environment, and candidates should be based in South Africa, preferably in Johannesburg or Cape Town.

Qualifications

• 6+ years of professional experience in data engineering or software engineering with a focus on data.
• Strong expertise in SQL, relational databases, and NoSQL databases.
• Experience with Data Vault modeling and ETL/ELT tools.

Responsibilities

• Design and maintain scalable data pipelines and ETL/ELT processes.
• Integrate structured and unstructured data from multiple sources.
• Collaborate with analysts and engineers to deliver data solutions.

Skills

Data architecture and pipeline development

Cloud and big data engineering

Collaboration with cross-functional teams

SQL and relational databases

ETL/ELT processes

Programming in Python, Scala, or Java

Education

Bachelor’s degree in Data Engineering or related field

Tools

AWS

Azure

GCP

Spark

Hadoop

Job description

About the Role

We are seeking a highly skilled Senior Data Engineer to join our dynamic data team. You will be responsible for designing, building, and maintaining robust and scalable data pipelines and architectures that enable analytics and business intelligence at enterprise scale. This role is ideal for professionals who thrive in complex data environments, value clean, efficient, and secure data engineering practices, and enjoy collaborating with cross-functional teams to deliver high-quality, reliable data solutions.

Key Responsibilities

1. Data Architecture & Pipeline Development

• Design, develop, and maintain scalable data pipelines and ETL/ELT processes
• Implement data modeling and architecture solutions including Data Vault, dimensional modeling, and normalized structures
• Optimize data storage and retrieval to support analytics, reporting, and machine learning workloads
• Integrate structured and unstructured data from multiple sources, including cloud and on-premises systems
• Develop and maintain data warehouses, data lakes, and cloud-native data platforms

Cloud & Big Data Engineering

• Work with cloud platforms such as AWS, Azure, or GCP for data storage, processing, and orchestration
• Implement and manage data processing frameworks such as Spark, Databricks, or Hadoop
• Develop scalable, automated, and reliable batch and real-time data workflows
• Ensure data pipelines meet performance, security, and compliance standards

Collaboration & Process Improvement

• Collaborate with data analysts, data scientists, software engineers, and business stakeholders to understand requirements and deliver solutions
• Mentor junior and intermediate data engineers, promoting best practices and code quality standards
• Participate in Agile/Scrum ceremonies, including sprint planning, stand‑ups, and retrospectives
• Continuously research and recommend new tools, technologies, and approaches to improve data engineering efficiency, reliability, and scalability

Requirements & Qualifications

Must-Have Skills

• 6+ years of professional experience in data engineering or software engineering with a focus on data
• Strong expertise in SQL, relational databases (PostgreSQL, MSSQL, MySQL), and NoSQL databases (MongoDB, Cassandra, DynamoDB)
• Experience with Data Vault modeling and implementation
• Hands‑on experience with ETL/ELT tools and processes
• Proficiency in programming/scripting languages such as Python, Scala, or Java
• Experience building data pipelines and architectures in cloud platforms (AWS, Azure, GCP)
• Strong understanding of data warehousing, data lakes, and cloud‑native storage solutions
• Experience with CI/CD for data pipelines and automation of data workflows

Highly Desirable Skills

• Experience with big data frameworks: Spark, Hadoop, Kafka
• Familiarity with orchestration tools such as Airflow, Prefect, or Azure Data Factory
• Knowledge of machine learning pipelines and integrating data for analytics
• Understanding of data governance, security, and compliance best practices
• Exposure to containerization (Docker, Kubernetes) in data workloads
• Strong communication skills to present complex data concepts to technical and business stakeholders

Education

• Bachelor’s degree, higher certificate, or diploma in Data Engineering, Computer Science, Information Systems, or a related field

Additional Information

• Contract Duration: 12 Months (with possibility of renewal)
• Salary: Negotiable
• Location Options: Johannesburg / Cape Town / Remote
• Must be located in South Africa or have a valid work permit for South Africa

For a more comprehensive list of opportunities, please visit our careers page: https://www.idbasesoftware.com/