Data Engineer

Two Roads Trading; Johannesburg; On-site; ZAR 400,000 - 600,000; Full time

Job summary

A data solutions company based in Johannesburg seeks a Data Engineer to design, build, and maintain data systems. You will be responsible for integrating data from various sources and ensuring its reliability. Ideal candidates have experience in building scalable data pipelines, proficiency in Python and SQL, and familiarity with cloud platforms. This role offers an opportunity to work closely with data analysts and other team members to support data-driven decision-making.

Qualifications

• Proven experience as a Data Engineer or ETL Developer.
• Strong expertise in building scalable data pipelines.
• Extensive experience with SQL and relational databases.

Responsibilities

• Design and optimize data architectures and pipelines.
• Implement ETL processes to consolidate data.
• Ensure data quality and integrity across systems.

Skills

Data pipeline development

SQL

Python

Database management

Data integration

Data quality assurance

Problem-solving

Collaboration

Education

Bachelor’s degree in computer science or equivalent

Tools

PostgreSQL

MySQL

AWS

Azure

Job description

Job Description

Job Summary :

A Data Engineer will be responsible for designing, building, and maintaining the infrastructure and systems that enable our organisation to collect, store, and analyse large volumes of data efficiently and securely. The candidate will develop data pipelines, integrate data from various sources, and ensure data is clean, reliable, and accessible for analysis.

Working closely with data analyst, and the team in our Efficiency department, the data engineers will focus on the architecture and performance of databases and data flows, helping ensure that data is well-organized and readily available for business intelligence and analytics purposes.

Key Responsibilities :

• Data Architecture & Design : Design, develop, and optimize scalable and efficient data architectures and data pipelines for ingesting, transforming, and loading data from diverse internal and external sources. A0WSA
• ETL / ELT Development : Build and maintain Extract, Transform, Load (ETL) or Extract, Load, transform (ELT) processes to consolidate data from various departmental systems and databases (e.g., spreadsheets, legacy systems, operational databases).
• Data Centralization : Implement solutions to centralize all available data, ensuring it is readily accessible and properly structured for analysis and reporting.
• Data Quality & Integrity : Implement robust data quality checks and validation procedures within the data pipelines to ensure the accuracy, completeness, and consistency of consolidated data. Develop monitoring tools to identify and address data integrity issues proactively.
• Database Management : Manage and optimize data storage solutions (e.g., relational databases, data warehouses, data lakes) to support analytical needs and efficient data retrieval.
• Integration : Work closely with the implementation team to ensure seamless data flow and integration between existing data sources and the new Zoho CRM system. This includes designing and implementing data migration strategies.
• Performance Optimization : Optimize data pipelines and database queries for performance, scalability, and cost-efficiency.
• Documentation : Create and maintain comprehensive documentation for data pipelines, data models, and data infrastructure.
• Collaboration : Collaborate with Data Analysts, business stakeholders, and IT teams to understand data requirements and ensure data solutions meet business needs.

Requirements

Qualifications and Desired Skills :

Bachelor’s degree in computer science, software engineering, data science, or a related field.

Proven experience as a Data Engineer, ETL Developer, or in a similar role.

High business acumen and data interpretation techniques and skills for business related scopes.

Strong expertise in designing, building, and maintaining scalable data pipelines.

Proficiency in programming languages commonly used for data engineering (e.g., Python, Java, Scala).

Extensive experience with SQL and database systems (e.g., PostgreSQL, MySQL, SQL Server, Oracle).

Experience with data warehousing concepts and technologies.

Familiarity with cloud data platforms (e.g., AWS, Azure, GCP) is a plus.

Demonstrated experience in implementing data quality and data governance best practices.

Experience with data integration tools and platforms.

Experience with CRM data, particularly Zoho CRM, and data migration strategies is highly desirable.

Strong problem-solving skills and attention to detail.

Individual must have high business acumen

Ability to work effectively in a team environment and communicate technical concepts clearly.

Requirements

Proven experience as a Data Engineer, ETL Developer, or in a similar role.

High business acumen and data interpretation techniques and skills for business related scopes.

Strong expertise in designing, building, and maintaining scalable data pipelines.

Proficiency in programming languages commonly used for data engineering (e.g., Python, Java, Scala).

Extensive experience with SQL and database systems (e.g., PostgreSQL, MySQL, SQL Server, Oracle).

Experience with data warehousing concepts and technologies.

Familiarity with cloud data platforms (e.g., AWS, Azure, GCP) is a plus.

Demonstrated experience in implementing data quality and data governance best practices.

Experience with data integration tools and platforms.

Experience with CRM data, particularly Zoho CRM, and data migration strategies is highly desirable.

Strong problem-solving skills and attention to detail.

Individual must have high business acumen

Ability to work effectively in a team environment and communicate technical concepts clearly.