Databricks Data Engineer - Build Scalable Data Pipelines

Decision Inc.; Johannesburg; On-site; ZAR 600,000 - 900,000; Full time

Benefits offered by this job

Competitive salary

Health benefits

Growth opportunities

Collaborative environment

Cutting-edge technologies

Job summary

Decision Inc is seeking a highly skilled Data Engineer in Johannesburg to design, build, and maintain data pipelines and platforms using Databricks. You will collaborate with data scientists, analysts, and business stakeholders to ensure data accuracy, efficiency, and scalability of the data infrastructure.

The role requires a minimum of 3 years of data engineering experience with Databricks, strong SQL and Python skills, and experience with cloud platforms (AWS/Azure/Google Cloud).

Qualifications

• Bachelor's degree required in Computer Science, Engineering or related field.
• Minimum 3 years of data engineering experience with a focus on Databricks.
• Strong SQL, Python and ETL skills are essential.
• Experience with cloud-based data platforms (AWS, Azure or Google Cloud).
• Knowledge of data modeling and data warehousing concepts.

Responsibilities

• Design, build and maintain data pipelines and platforms using Databricks.
• Collaborate with data scientists, analysts and stakeholders to understand data requirements.
• Develop data quality and monitoring processes to ensure accuracy and reliability of data.
• Optimize data pipelines for performance and scalability.
• Troubleshoot and resolve data pipeline issues promptly.
• Stay updated with data engineering trends and propose infrastructure improvements.
• Mentor and provide technical guidance to junior data engineers.

Skills

SQL

Python

ETL processes

Data modeling

Education

Bachelor's degree in Computer Science, Engineering or related field

Tools

Databricks

AWS

Azure

Google Cloud

Job description

Decision Inc is seeking a highly skilled Data Engineer in Johannesburg to design, build, and maintain data pipelines and platforms using Databricks. You will collaborate with data scientists, analysts, and business stakeholders to ensure data accuracy, efficiency, and scalability of the data infrastructure.

The role requires a minimum of 3 years of data engineering experience with Databricks, strong SQL and Python skills, and experience with cloud platforms (AWS/Azure/Google Cloud).