Databricks Data Engineer

Tenth Revolution Group; Cape Town; Sur place; ZAR 700 000 - 1 100 000; Plein temps

Résumé du poste

Tenth Revolution Group in Cape Town seeks an experienced Azure Data Engineer to design, build, and optimize scalable cloud data solutions on Microsoft Azure. You will work on batch and real-time data pipelines to support analytics, reporting, and advanced use cases.

Join a growing data team and collaborate with stakeholders, data analysts, and platform teams to deliver reliable, secure, high-performing data architectures across data lake and warehouse environments. In-office presence is required.

Qualifications

• Bachelor's degree in Computer Science, Engineering, Data Science, or related field.
• 3–5 years hands-on data engineering experience.
• Strong expertise with Azure data services and cloud security.

Responsabilités

• Design, develop, and maintain ETL/ELT pipelines using Azure Data Factory.
• Build and optimize large-scale data processing solutions with Azure Databricks (Spark).
• Implement real-time data ingestion with Azure Event Hubs.
• Collaborate with stakeholders to deliver trusted datasets for analytics and BI.
• Ensure data quality, governance, and secure, scalable data architectures.

Connaissances

ETL/ELT design patterns

Data modeling

DevOps practices & CI/CD

Stakeholder engagement

Agile delivery

Formation

Bachelor's degree in Computer Science, Engineering, Data Science, or related field

Outils

Azure Data Factory

Azure Databricks

Azure Event Hubs

Delta Lake

Spark

SQL

Python

Description du poste

Job Title: Data Engineer (Azure)

Location: Cape Town, Claremont

In-office Expectation: 4 days in office

Position Type: Full-time / Permanent

Reports To: Head of Data

About the Role

We are seeking a skilled Azure Data Engineer with 5 years of experience to join our growing data team. This role focuses on designing, building, and optimizing scalable cloud-based data solutions using Microsoft Azure technologies. You will be responsible for batch and real-time data ingestion, transformation, and integration to support analytics, reporting, and advanced data use cases.

You will work closely with business stakeholders, data analysts, and platform teams to deliver reliable, secure, and high-performing data pipelines.

Key Responsibilities

Data Engineering & Pipeline Development

• Design, develop, and maintain ETL/ELT pipelines using Azure Data Factory (ADF).
• Build and optimize large-scale data processing solutions using Azure Databricks (Spark).
• Implement real-time data ingestion using Azure Event Hubs.
• Develop and maintain scalable data models for analytics and reporting.
• Perform data transformation, cleansing, and enrichment processes.

Cloud & Data Platform Engineering

• Support and enhance Azure-based data lake and data warehouse architectures.
• Optimize data storage, partitioning, and performance strategies.
• Ensure high availability, scalability, and cost-efficiency of data solutions.
• Automate workflows and support CI/CD for data pipelines.

Data Integration & Streaming

• Integrate structured and unstructured data from multiple enterprise systems.
• Design solutions for both batch and streaming data pipelines.
• Collaborate with integration teams on event-driven architectures.

Data Quality, Governance & Security

• Implement data validation, monitoring, and reconciliation processes.
• Apply data governance and security best practices across Azure services.
• Document data lineage, transformations, and architecture components.

Collaboration & Delivery

• Translate business requirements into scalable technical solutions.
• Partner with analytics and BI teams to deliver trusted datasets.
• Participate in agile delivery cycles and code reviews.

Qualifications & Experience

• Bachelor’s degree in Computer Science, Engineering, Data Science, or related field.
• 3–5 years of hands‑on experience in data engineering roles.

Strong expertise in:

• Azure Data Factory
• Azure Databricks
• Azure Event Hubs
• SQL (advanced level)
• Python (preferred)
• Experience designing data lakes and data warehouse solutions.
• Strong understanding of ETL/ELT design patterns.
• Experience working with Azure cloud services and security models.
• Knowledge of data modeling (dimensional and normalized models).

Desired Skills

• Experience with Delta Lake and Spark optimization.
• Familiarity with DevOps practices and CI/CD pipelines.
• Exposure to event-driven architecture concepts.
• Strong troubleshooting and performance tuning skills.
• Excellent communication and stakeholder engagement abilities.

Why Join Us?

• Work in a modern Azure-first data ecosystem.
• Opportunity to build both batch and real-time data solutions.
• Exposure to enterprise-scale data architecture initiatives.
• Supportive and forward-thinking data leadership.