Senior Databricks Data Engineer

Confluent; Cape Town; Hybrid; ZAR 650,000 - 900,000; Full time

Benefits offered by this job

Medical aid

Retirement annuity

Bursaries and Insurance

Job summary

Confluent Services, South Africa’s leading debt management company, seeks a Data Engineer to design, build, and maintain data pipelines across on‑prem and cloud platforms. You’ll work with Azure Databricks and Data Factory within a growing Data Team, influencing architecture and data workflows.

You will collaborate with analysts, data scientists, and stakeholders to ensure data quality and enable analytics and ML initiatives.

Qualifications

• Bachelor's degree in CS/Engineering or related field.
• Minimum 3 years as a data engineer or similar role.
• Experience with Azure services (ADF, Databricks) preferred.
• Knowledge of SQL Server and Azure Data Lake Storage.
• Strong communication and teamwork abilities.
• Attention to detail and data problem-solving skills.

Responsibilities

• Contribute to architecture and design of the Confluent Data Platform.
• Develop and support scalable data pipelines on Azure.
• Ensure data quality, security and integrity across pipelines.
• Collaborate with data product owners, analysts and scientists.
• Assist in data initiatives and support technical teams.

Skills

Data pipelines

ETL processes

Azure

Azure Data Factory

Azure Databricks

SQL

Data Lake

Teamwork

Attention to detail

Problem solving

Education

Bachelor's degree in Computer Science or Engineering

Tools

Azure

Azure Data Factory

Azure Databricks

Microsoft Fabric Data Engineering

SQL Server / Data Lake Storage

Job description

(Hybrid/Pinelands, Cape Town)

Confluent Services, South Africa’s largest and multi-award winning debt management company, has a position available for aData Engineer.

This individual will join the Data Team and will be accountable for data engineering design, data ingestion and support across on-premise and cloud-based platforms.

What’s in it for you?

Being South Africa’s largest company of its kind, Confluent is continuously growing, with no signs of stopping. At Confluent, we build on current-generation platforms like Databricks and Microsoft Fabric, giving you hands‑on exposure to tools and patterns that are shaping the future of data engineering. This is not a legacy maintenance role, you’ll be involved in designing, building, and evolving data solutions from the ground up, with real room for creativity and engineering judgement.

You’ll work alongside a team of high caliber senior data engineers, where knowledge sharing is part of how we operate. Technical discussions are encouraged, opinions are respected, and decisions are driven by merit rather than hierarchy. Our team is actively involved in AI and machine learning initiatives, giving you exposure beyond traditional data pipelines.

We value clean architecture, thoughtful design, and creativity. You will have the opportunity to influence how things are built, contribute to architectural decisions, and help shape the evolution of our data platform.

Culturally, we are a diverse, respectful, and collaborative team. We invest in people, support continuous learning, and create an environment where engineers can grow both technically and professionally. Whether you want to deepen your engineering expertise or broaden your impact across the business, you’ll find room to do so here.

Beyond the work itself, Confluent offers a stable, growing environment where the data you build helps real South Africans improve their financial outcomes, giving purpose and impact to the systems you design.

Other key benefits:

• We have a vibrant office in a lively part of the city, meaning you have easy access to stores and outlets if you need.
• We offer company benefits – medical aid, retirement annuity, Bursaries and Insurance
• We have a culture of growth, meaning you have access to move both upwards or across in the business if you want to.

Requirements

Candidates must be able to demonstrate:

• Minimum 3 years experience as a data engineer or in a similar role, designing and implementing data pipelines, data integration and ETL processes.
• Practical experience in Azure, Azure Data Factory and Azure Databricks is preferred.
• At least 3 years of experience in Information Technology – specifically in technical data roles e.g. data engineering, data analytics or data science.
• Knowledge of data management and storage solutions, including SQL Server Database, Azure Data Lake Storage or similar.
• Having good communication skills and being able to operate in a team environment.
• Excellent attention to detail and problem‑solving skills related to data, data structures and algorithms.

Key responsibilities will include:

• Contribute to the architecture and design of the Confluent Data Platform and ingestion patterns.
• Implement and support the scalable data solutions on the Microsoft Azure cloud platform, including Azure Databricks.
• Develop, maintain and optimise robust production data pipelines for data ingestion, processing, and transformation to support analytics, machine learning and other downstream data‑driven services.
• Ensure data security, quality and integrity across all of the pipelines and related data processes.
• Collaborate with the data product owner, analysts, data scientists, and business stakeholders to understand data requirements and implement effective data workflows.
• Assist in data initiatives and support technical teams including architects, analysts, software developers, and data scientists.

Qualifications:

• Bachelor's degree in Computer Science, Engineering, or a related field.
• Certifications in Azure, Microsoft Fabric Data Engineering, Azure Databricks Data Engineering or similar or equivalent practical experience.