Kafka & Flink Data Engineer

Pbt Group; Johannesburg; On-site; ZAR 600 000 - 800 000; Full time

Job summary

A technology solutions company in Johannesburg is seeking an experienced Data Engineer to design and maintain high-performance data pipelines. The ideal candidate will have strong experience with Apache Flink and Kafka, and proficiency in Python, Java, or Scala. Collaboration with data teams and a proactive, adaptable approach are essential. This role involves ensuring the reliability and scalability of data processing frameworks.

Qualifications

• Proven expertise in building and optimizing data processing systems.
• Experience with distributed data processing and event streaming.
• Strong background in modern data architecture design.

Responsibilities

• Design, develop, and maintain high-performance data pipelines and streaming solutions.
• Integrate and optimize data flows across multiple systems.
• Ensure scalability, reliability, and security of data processing frameworks.

Skills

Apache Flink

Apache Kafka

Spring Boot

Python

Java

Scala

CI/CD Pipelines

Git

Jenkins

Tools

AWS

Kubernetes

Docker

Job description

We are looking for an experienced Data Engineer with a strong background in building and optimising data processing systems. The ideal candidate will have proven expertise in distributed data processing, event streaming, and modern data architecture design.

Technical Requirements:

• Strong experience in Apache Flink for real-time stream processing
• Proficiency in Apache Kafka for event-driven data pipelines
• Experience with Spring Boot (microservices development and integration)
• Solid understanding of data integration, data modelling, and data quality principles
• Strong development background in Python, Java, or Scala
• Hands-on experience working within cloud or containerised environments (e.g., AWS, Kubernetes, Docker advantageous)
• Experience with CI/CD pipelines and version control (Git, Jenkins, etc.)
• Good understanding of relational and non-relational databases

Soft Skills:

• Strong analytical and problem-solving ability
• Collaborative and proactive approach within agile teams
• Good communication and documentation skills
• Ability to adapt to fast-changing technology environments

Deliverables:

• Design, develop, and maintain high-performance data pipelines and streaming solutions
• Integrate and optimise data flows across multiple systems
• Ensure scalability, reliability, and security of data processing frameworks
• Collaborate with architects and data teams to deliver robust data solutions