Data Engineer (Integration for Data Science Team)

PBT Group; Johannesburg; On-site; ZAR 600 000 - 800 000; Full time

Job summary

A technology services company is seeking a skilled Data Engineer to design and optimize data pipelines supporting advanced analytics. The role involves collaboration with data teams to ensure data quality and integration into AWS and Databricks environments. Candidates should have over 4 years of experience, strong AWS and Python skills, and familiarity with data orchestration tools. A degree in a related field is preferred.

Qualifications

• 4+ years’ experience as a Data Engineer, preferably in financial services or insurance.
• Advanced proficiency in Python for ETL/ELT development and automation.
• Familiarity with AWS services.

Responsibilities

• Design, build, and maintain robust data pipelines.
• Integrate data from multiple source systems into AWS and Databricks environments.
• Implement data quality and validation frameworks.

Skills

Data pipeline design and maintenance

Collaboration with data scientists and analysts

AWS (Lambda, S3, Glue, Redshift, EMR)

Python for ETL/ELT development

SQL and data modelling

Data orchestration tools (e.g., Airflow)

Agile methodologies

Education

Degree in Computer Science or related field

Tools

Databricks

Salesforce

TIA

Job description

PBT Group is seeking a skilled Data Engineer to join a growing Data Science team responsible for designing, developing, and optimising data pipelines and integration frameworks to support advanced analytics and machine learning initiatives. The successful candidate will play a key role in ensuring seamless data flow, scalability, and reliability across cloud and on-premise environments.

Key Responsibilities

• Design, build, and maintain robust data pipelines for ingestion, transformation, and delivery of structured and unstructured data.
• Collaborate closely with data scientists, analysts, and business stakeholders to ensure data solutions meet analytical and operational needs.
• Optimise existing ETL/ELT processes for performance, scalability, and cost efficiency.
• Integrate data from multiple source systems (including APIs, databases, and third-party applications) into AWS and Databricks environments.
• Implement data quality, monitoring, and validation frameworks to ensure high standards of data integrity.
• Support the deployment and operationalisation of machine learning models and data products.
• Work in Agile or DevOps-driven environments using CI/CD pipelines and version control tools.

Skills & Experience Required

• 4+ years’ experience as a Data Engineer, preferably within financial services or insurance.
• Strong expertise in AWS (Lambda, S3, Glue, Redshift, EMR) and Databricks.
• Advanced proficiency in Python for ETL/ELT development and automation.
• Experience with SQL and data modelling for analytical and operational databases.
• Exposure to data orchestration tools (e.g., Airflow, Step Functions) and streaming technologies (Kafka advantageous).
• Familiarity with Salesforce, TIA, or other insurance-related data systems would be highly beneficial.
• Experience working in Agile environments with Git-based version control and CI/CD tools.

Preferred Qualifications

• Degree in Computer Science, Information Systems, Data Engineering, or related field.
• AWS Certified Data Engineer / Databricks Certified Data Engineer Associate (advantageous).