Data Engineer

Parvana; Remote; ZAR 600 000 - 800 000; Full time

Job summary

A reliable software solutions provider in South Africa is seeking a remote Data Analytics Engineer. The role involves analyzing complex customer data, developing scalable ETL pipelines, and automating data validation processes. The ideal candidate will possess strong coding skills in Python, PySpark, and SQL, along with experience in cloud environments like Azure or AWS. If you have a proven track record in data engineering, this position offers a unique opportunity to advance in a dynamic environment.

Qualifications

• 5+ years of experience as a Data Analytics Engineer.
• Strong coding skills in Python, PySpark, Notebook, and SQL.
• Proven experience building scalable ETL pipelines in cloud environments.

Responsibilities

• Analyze complex customer data to determine integration needs.
• Develop and test scalable data integration/transformation pipelines.
• Automate and maintain data validation and quality checks.

Skills

Python

PySpark

SQL

Data integration

Agile/Scrum

Data quality concepts

Tools

Databricks

Delta Lake

CI/CD

Terraform

Pulumi

Azure

AWS

Job description

This is a remote position.

About our client: Our client develops and supports software and data solutions across a variety of industries. They want you to get ahead of the market and stay there. They offer a combination of plug and play products that can be integrated with existing systems and processes and can also be customised to client needs. Their capabilities extend to big data engineering and bespoke software development, solutions are available as both cloud-based and hosted.

What you will be doing:

• Analyzes complex customer data to determine integration needs.
• Develops and tests scalable data integration/transformation pipelines using PySpark, SparkSQL, and Python.
• Contributes to the codebase through coding, reviews, validation, and complex transformation logic.
• Automates and maintains data validation and quality checks.
• Collaborates with FPA, data engineers, and developers to align solutions with financial reporting and business objectives.
• Participates in solution architecture and technical discussions, refining user stories and acceptance criteria.
• Utilizes modern data formats/platforms (Parquet, Delta Lake, S3/Blob Storage, Databricks).
• Partners with the product team to ensure accurate customer data reflection and provide feedback based on data insights.

What our client is looking for:

• A Data Analytics Engineer with 5+ years of experience.
• Must have strong Python, PySpark, Notebook, and SQL coding skills, especially with Databricks and Delta Lake.
• Proven ability to build and deploy scalable ETL pipelines to cloud production environments using CI/CD.
• Experience with Agile/Scrum, data quality concepts, and excellent communication is essential.
• Cloud environment (Azure, AWS) and Infrastructure as Code (Terraform, Pulumi) experience beneficial.
• Telecoms industry or consulting experience, plus accounting knowledge, is a plus.

Job ID: J106998

For a more comprehensive list of opportunities that we have on offer, do visit our website - https://www.parvana.co.uk/careers