Junior Data Engineer

BETSoftware; Gauteng; On-site; ZAR 500,000 - 800,000; Full time

Job summary

BETSoftware is seeking a data engineer in Gauteng with foundational SQL, data querying, and relational database skills. The role focuses on building and supporting batch data pipelines, data ingestion, transformation, and data prep across the enterprise data platform.

You will collaborate with engineers to monitor pipelines, resolve data issues, and contribute to platform growth with warehousing concepts, orchestration tools, and CI/CD practices.

Qualifications

• Degree, diploma, or relevant certification in IT, CS, Engineering, or Information Systems.

Responsibilities

• Build, maintain, and support batch data pipelines under guidance.
• Assist ingestion, transformation, and data prep across the data platform.

Skills

SQL

Data querying

Relational databases

Data pipelines

ETL/ELT

Debugging

Git

Python exposure

Data warehousing

CI/CD

Documentation

Communication

Problem solving

Education

Degree, diploma, or relevant IT certification

Tools

Airflow

Azure DevOps

Git

Spark/PySpark

Kafka/Redpanda

ClickHouse

Iceberg

Delta Lake

SQL Server Agent

Job description

Job Description

• Proficiency in SQL and data querying fundamentals.
• Understanding of relational databases and structured data management.
• Knowledge of data pipelines, ETL/ELT processes, and batch data processing concepts.
• Ability to troubleshoot, debug, and resolve technical issues.
• Familiarity with version control systems, particularly Git.
• Exposure to Python or similar scripting languages for data-related tasks.
• Understanding of data warehousing, dimensional modelling, and analytical data concepts.
• Familiarity with workflow orchestration tools such as Airflow or SQL Server Agent.
• Knowledge of cloud and on-premises storage concepts.
• Awareness of distributed processing, streaming architectures, and lakehouse technologies.
• Exposure to modern data engineering tools and platforms, including Spark, PySpark, Kafka, Redpanda, ClickHouse, Iceberg, and Delta Lake.
• Understanding of monitoring, observability, alerting, and operational support practices for data workloads.
• Familiarity with CI/CD pipelines, deployment automation, and engineering best practices.
• Ability to work effectively with technical documentation, runbooks, and team standards.
• Strong analytical, problem-solving, and communication skills.
• Ability to collaborate with technical and non-technical stakeholders.
• High attention to detail when working with business-critical data.
• Strong accountability, ownership mindset, and commitment to continuous learning and professional growth in data engineering.

Job Responsibilities

Data Pipeline Support & Development

• Help build, maintain, and support batch data pipelines under the guidance of more experienced engineers.
• Assist with ingestion, transformation, and data preparation tasks across the enterprise data platform.
• Write and maintain SQL queries, scripts, and basic processing logic for data workflows.
• Support the improvement of existing pipelines by helping resolve defects, inefficiencies, and data issues.

Data Platform Operations

• Monitor scheduled pipelines and help investigate failures, alerts, and data discrepancies.
• Assist in validating data loads and checking that datasets are complete, accurate, and available when expected.
• Help maintain documentation for data flows, processes, dependencies, and operational procedures.
• Contribute to testing, deployment, and support activities for data-related changes.

Learning & Platform Growth

• Learn data engineering standards, patterns, and tools used across the team.
• Work with senior and intermediate engineers to understand platform architecture, pipeline design, and production support practices.
• Contribute to continuous improvement initiatives through automation, clean-up work, and better documentation.
• Build technical capability over time in areas such as orchestration, data quality, lakehouse concepts, streaming, and scalable data processing.

Collaboration & Delivery

• Work with BI, analytics, software engineering, and business teams to understand data requirements and support delivery outcomes.
• Participate in team planning, estimation, development, testing, and release activities.
• Ask for guidance when needed and apply feedback constructively.
• Contribute to a collaborative engineering culture through clear communication and reliable follow-through.

Tech Environment

The platform may include a combination of established and modern technologies such as, SQL Server, Python, Airflow, Azure DevOps / Git, Spark or PySpark, Object storage, Iceberg, Kafka, or Redpanda, ClickHouse or similar columnar analytical stores and CI/CD tooling and engineering workflows.

Qualifications

• Degree, diploma, or relevant certification in IT, Computer Science, Engineering, Information Systems, or a related technical discipline.
• Minimum 1+ years proven experience in data engineering, software development, ETL/ELT, database development, or a related technical role.
• Foundational SQL skills, including writing queries, joining datasets, and working with basic transformations.
• Some exposure to Python, data processing, scripting, or automation.
• Understanding of relational databases and structured data concepts.
• Willingness to learn modern data platform tools, engineering practices, and production support processes.

Technical Skills

You have a basic working understanding of:

• SQL and data querying fundamentals.
• Relational databases and structured data handling.
• Data pipelines, ETL/ELT, or batch-processing concepts.
• Debugging and troubleshooting technical issues.
• Version control concepts such as Git.

Exposure To The Following Would Be Advantageous

• Python or similar scripting languages.
• Data warehousing, dimensional modelling, or analytical data concepts.
• Workflow orchestration tools such as Airflow or SQL Server Agent.
• Cloud or on-premises object storage concepts.
• Distributed processing, streaming, or lakehouse concepts.
• Tools such as Spark, PySpark, Kafka, Redpanda, ClickHouse, Iceberg, or Delta Lake.

Platform & Engineering Practices

• Interest in learning monitoring, alerting, observability, and operational support for data workloads.
• Exposure to CI/CD, deployment automation, or engineering workflows is beneficial.
• Comfort working with documentation, runbooks, and team standards.

Personal Attributes

• Positive learning mindset and willingness to ask questions.
• Strong sense of accountability and follow-through.
• Good problem-solving and communication skills.
• Comfortable collaborating with both technical and non-technical stakeholders.
• Attention to detail and willingness to work carefully with business-critical data.
• Motivation to build a long-term career in data engineering.

Living the Spirit

• Engages in cross-functional collaboration and problem solving while contributing to an inclusive team culture.
• Supports a culture of adaptability and shared accountability across the department and wider business.
• Shows up authentically and contributes to team success by working effectively with diverse colleagues and perspectives.
• Approaches challenges as opportunities to learn, improve, and help others grow.