Hybrid Azure Databricks Data Engineer — Shape Data Platforms

Confluent; Cape Town; Hybrid; ZAR 600,000 - 900,000; Full time

Benefits offered by this job

Vibrant office in Cape Town

Medical aid

Retirement annuity

Bursaries and Insurance

Growth opportunities

Job summary

Confluent Services, South Africa’s largest debt management company, is hiring a Data Engineer to join the Data Team in a hybrid role based in Pinelands, Cape Town. You will design, build and operate data pipelines across on-premise and cloud platforms, with exposure to Databricks and Microsoft Fabric ecosystems.

You’ll work with senior engineers, contribute to architectural decisions, ensure data quality and security, and collaborate with analysts, data scientists and stakeholders to deliver

Qualifications

• 3 years of data engineering or similar role building data pipelines.
• Experience with Azure, Azure Data Factory and Azure Databricks preferred.
• Knowledge of SQL Server or Azure Data Lake Storage.
• Strong teamwork and communication skills.
• Detail-oriented with data structures and algorithms.

Responsibilities

• Contribute to architecture and design of Confluent Data Platform and ingestion patterns.
• Develop and optimize production data pipelines on the Microsoft Azure cloud platform.
• Ensure data security, quality and integrity across all pipelines.
• Collaborate with data product owners, analysts, data scientists and stakeholders.
• Assist in data initiatives and support technical teams including architects and developers.

Skills

Azure

ETL

Data pipelines

SQL

Communication

Education

Bachelor's degree in Computer Science or Engineering

Tools

Azure Databricks

Azure Data Factory

Job description

Confluent Services, South Africa’s largest debt management company, is hiring a Data Engineer to join the Data Team in a hybrid role based in Pinelands, Cape Town. You will design, build and operate data pipelines across on-premise and cloud platforms, with exposure to Databricks and Microsoft Fabric ecosystems.

You’ll work with senior engineers, contribute to architectural decisions, ensure data quality and security, and collaborate with analysts, data scientists and stakeholders to deliver