Data Engineer

X, bigly labs; Johannesburg; On-site; ZAR 600,000 - 1,200,000; Full time

Job summary

X, bigly labs in Johannesburg is building a modern data platform to power analytics, ML, and AI initiatives for healthcare outcomes. We seek a Data Engineer to lead robust data pipelines and governance, enabling governed data sharing with external partners and supporting business users across analytics and AI workloads.

You will work with analysts, scientists, and AI engineers to ensure data quality, privacy, and compliant data flows while documenting architecture and standards for

Qualifications

• Bachelor’s degree in Computer Science, Data Engineering, Information Systems, or a related field.
• Certification in cloud platforms (Azure, AWS, GCP), Databricks, or data engineering.
• 6+ years of experience in data engineering or software development.
• 3+ years of hands-on experience with Databricks and implementing Medallion Architecture.
• Proven experience in developing data platforms that support data science, data analytics and AI workloads.
• Strong proficiency in Python, SQL, Spark, and cloud-native data services (e.g., Azure Data Lake, AWS S3, GCP BigQuery).
• Experience with CI/CD pipelines, data governance, and data quality frameworks.
• Solid understanding of data science workflows, model lifecycle, and AI concepts.

Responsibilities

• Implement data lakehouse architecture principles by designing Bronze, Silver, and Gold layers to optimise data quality, lineage, and usability for downstream consumers.
• Enable secure and governed data sharing with external partners by applying access controls, encryption, and data contracts to ensure compliance, trust, and seamless collaboration.
• Collaborate with data analysts, data scientists, AI engineers and data quality analysts through regular engagement, feedback sessions, and shared development environments to ensure the platform meets analytical and modelling needs effectively.
• Ensure data integrity, privacy, and compliance by implementing automated data quality checks, lineage tracking, and governance frameworks to maintain trust and meet regulatory standards (e.g., POPIA, GDPR).
• Document technical architecture, data flows, and engineering standards using version-controlled repositories and collaborative tools to support maintainability, onboarding, and cross-functional transparency.

Skills

Python

SQL

Spark

Databricks

Cloud (Azure/GCP/AWS)

Data governance

CI/CD

Data quality

Data modeling

Education

Bachelor’s degree in CS/Data Engineering

Tools

Databricks

Airflow

Azure Data Lake

AWS S3

BigQuery

Job description

We are X, bigly labs, Dis-Chem’s high-performance innovation hub; a place where bold ideas meet data, design, and radical customer focus to reimagine what healthcare can be.

Our mission is to power the future of healthcare by lowering costs, unlocking new possibilities, and improving access to healthcare for all South Africans. We do this through cutting-edge digital solutions that deliver smarter, more human, and truly patient-focused experiences. Here, your work is not limited to a whiteboard; it becomes real. It drives decisions, shapes experiences, and moves healthcare forward.We are driven by one big question: How do we use data and today’s technology to build breakthroughs to better health?

There’s only one question left: are you ready to make healthcare smarter, and actually mean it?

We are seeking Data Engineer to lead the development of a modern, scalable, and secure data platform that supports enterprise-wide analytics, machine learning, and AI initiatives. The Senior Data Engineer will be responsible for building robust data pipelines and enabling governed data sharing with external partners. This role will work closely with data analysts, data scientists, and AI engineers to ensure the platform enables self-analytics by business users and advanced analytical, modelling and AI needs.

WHAT WE’RE LOOKING FOR?

Minimum

• Bachelor’s degree in Computer Science, Data Engineering, Information Systems, or a related field.
• Certification in cloud platforms (Azure, AWS, GCP), Databricks, or data engineering.
• 6+ years of experience in data engineering or software development.
• 3+ years of hands-on experience with Databricks and implementing Medallion Architecture.
• Proven experience in developing data platforms that support data science, data analytics and AI workloads.
• Strong proficiency in Python, SQL, Spark, and cloud-native data services (e.g., Azure Data Lake, AWS S3, GCP BigQuery).
• Experience with CI/CD pipelines, data governance, and data quality frameworks.
• Solid understanding of data science workflows, model lifecycle, and AI concepts.

Preferred Qualifications

• Databricks Certified Data Engineer Associate or Professional.
• Certification in Machine Learning or AI (e.g., Azure AI Engineer Associate).
• Experience in healthcare, retail, or insurance data ecosystems.

WHAT YOU WILL BE DOING?

• Implement data lakehouse architecture principles by designing Bronze, Silver, and Gold layers to optimise data quality, lineage, and usability for downstream consumers.
• Enable secure and governed data sharing with external partners by applying access controls, encryption, and data contracts to ensure compliance, trust, and seamless collaboration.
• Collaborate with data analysts, data scientists, AI engineers and data quality analysts through regular engagement, feedback sessions, and shared development environments to ensure the platform meets analytical and modelling needs effectively.
• Ensure data integrity, privacy, and compliance by implementing automated data quality checks, lineage tracking, and governance frameworks to maintain trust and meet regulatory standards (e.g., POPIA, GDPR).
• Document technical architecture, data flows, and engineering standards using version-controlled repositories and collaborative tools to support maintainability, onboarding, and cross-functional transparency.

WHO YOU ARE?

• Have an advanced knowledge of Databricks and Apache Spark.
• Analytical thinking and problem-solving
• Balances multiple initiatives and deadlines with structured focus
• Helps coordinate timelines, deliverables, and interdependencies across teams
• Demonstrates the ability to navigate ambiguity with confidence and composure.

Our values show up in how we think, build, and make choices that make an impact. We are building a culture that drives progress; one that fixes friction, chases what changes, and owns the outcome. We design with tomorrow in mind, ask sharper questions, and answer them with care, urgency, and systems that scale.

Think you’ve got energy, curiosity, and guts? Come hold the space, make it matter, and own a breakthrough.