Senior Azure Data Engineer

PBT Group; Johannesburg; On-site; ZAR 400 000 - 500 000; Full time

Job summary

A technology solutions company in Johannesburg is seeking a Senior Data Engineer to enhance their data platform project. The role involves designing and optimizing data pipelines in the Azure ecosystem, using tools like Azure Data Factory and Spark. Ideal candidates should have hands-on experience in data engineering and strong leadership skills to mentor junior engineers, ensuring data quality and governance standards are met.

Qualifications

• Proven background in designing and building data pipelines.
• Hands-on technical expertise in the Azure ecosystem.
• Experience in mentoring junior team members.

Responsibilities

• Design, develop, and maintain data pipelines in Azure.
• Optimize workflows for performance using Spark.
• Collaborate across teams to meet data requirements.
• Implement data quality frameworks.

Skills

Azure Data Factory

Apache Spark

Data Lake

ETL/ELT design

Data modelling

Problem-solving

Communication

Tools

Azure Synapse

Databricks

SQL Server

Power BI

Job description

PBT Group is seeking an experienced Senior Data Engineer to accelerate delivery on an existing data platform project and mentor a team of junior engineers. The successful candidate will have a proven background in designing, building, and optimising large-scale data pipelines within the Azure ecosystem, particularly leveraging Azure Data Factory, Spark, and Data Lake technologies.

This role requires both hands-on technical expertise and strong leadership capabilities to guide best practices, improve performance, and ensure data quality and governance standards are met.

Key Responsibilities

• Design, develop, and maintain robust data pipelines and ETL processes within the Azure environment.
• Optimise data workflows for scalability, reliability, and performance using Spark and Azure Data Lake.
• Collaborate with cross-functional teams (data analysts, architects, and business stakeholders) to translate requirements into efficient data solutions.
• Implement data quality frameworks and monitor data integrity across systems.
• Provide technical guidance, mentorship, and peer reviews for junior and intermediate data engineers.
• Troubleshoot and resolve data-related issues, ensuring smooth operational continuity.
• Participate in solution design discussions and contribute to architectural decisions.

Technical Skills & Experience

• Core Technologies:
  • Azure Data Factory (ADF)
  • Azure Data Lake / Data Lakehouse concepts
  • Apache Spark / PySpark
• Data Engineering Expertise:
  • ETL/ELT design and orchestration
  • Data modelling and optimisation for analytical workloads
  • Experience with structured and unstructured data sources
• Additional Experience (advantageous):
  • Azure Synapse, Databricks, SQL Server, or Power BI integration
  • CI/CD pipelines and DevOps data deployment practices
• Soft Skills:
  • Strong problem-solving and analytical skills
  • Proven ability to work independently and collaboratively
  • Excellent communication and mentorship abilities

Ideal Candidate Profile

A technically skilled, self-driven professional who thrives in dynamic environments and enjoys improving data systems and empowering teams. You should bring a strong Azure data engineering foundation, hands-on Spark experience, and a genuine passion for data performance and quality.

In order to comply with the POPI Act, for future career opportunities, we require your permission to maintain your personal details on our database. By completing and returning this form you give PBT your consent

If you have not received any feedback after 2 weeks, please consider you application as unsuccessful.