Senior Data Engineer

Frostbyte; Cape Town; On-site; ZAR 500 000 - 700 000; Full time

Job summary

A leading technology firm in Cape Town is seeking a mid-senior level data engineer to develop and maintain SQL databases and data engineering pipelines in the cloud (Azure). You will ensure DevOps compliance, provide mentorship to junior engineers, and troubleshoot issues within production. Ideal candidates should have a B.Sc. in a relevant field and experience with Python, Azure Data Factory, and debugging techniques. This full-time position offers a great opportunity for professional growth.

Qualifications

• B.Sc. in Electronic and/or Computer Engineering or similar qualification.
• Mid-Senior level experience in data engineering.
• Ability to debug and troubleshoot technical issues.

Responsibilities

• Develop and maintain SQL Databases.
• Create data engineering and analysis pipelines in the cloud (Azure).
• Ensure DevOps compliance for data pipelines and workloads.
• Maintain Power BI reports.
• Document pipelines and databases.
• Provide support to the Technical Operations team.

Skills

Python

pyspark

Data Lake experience

Azure Data Factory

Debugging

Technical documentation

Interpersonal skills

Communication skills

Education

B.Sc. in Electronic and/or Computer Engineering or similar

Tools

Azure Data Lake gen 2

Databricks

Azure Synapse

Job description

Referrals increase your chances of interviewing at FROSTBYTE by 2x

Responsibilities

Development and maintenance of SQL Databases

Development of data engineering and data analysis pipelines and solutions in the cloud (Azure)

Ensure DevOps compliance for all data pipelines and workloads

Ensure development and maintain Power BI reports

Documentation of pipelines and databases

Troubleshooting and support to our Technical Operations team when dealing with production issues

Contribute to application and systems design in collaboration with the Software Architects.

Provide mentorship and support to junior data engineers

Qualifications

B.Sc.

(Electronic and / or Computer Engineering) or similar qualification.

Required Skill set

Python and preferably pyspark experience

Data Lake experience (Azure Data Lake gen 2 experience preferred)

Azure Data Factory (ADF) experience or similar

Debugging and troubleshooting skills

Experience in writing technical documentation (architectural diagrams, release notes, etc.)

Good inter-personal and communication skills

Additional Nice-to-have Skills

Data Warehouse experience – especially Azure Synapse

Databricks or Spark experience, with Azure Databricks being preferred

Data Science and AI / ML experience would be beneficial

Seniority level

Mid-Senior level

Employment type

Full-time

Job function

Information Technology