Data Alchemist: Hybrid Cloud Data Engineer

Calibrate People; Johannesburg; Hybrid; ZAR 700,000 - 1,100,000; Full time

Job summary

Calibrate People is seeking a Data Engineer to join our Technology Team in a hybrid Johannesburg setting. The role focuses on building scalable data pipelines and ETL workflows across cloud and on-prem systems, collaborating with data scientists and analysts to deliver actionable insights.

You will leverage tools like AWS/Azure, Spark, and SQL, while ensuring data quality, security, and governance. Strong communication and problem-solving are essential for success in this fast-paced environment.

Qualifications

• Bachelor's degree from an accredited university.
• 3+ years hands-on data engineering; finance sector experience is a plus.
• Experience with on-premises and cloud data solutions.
• Familiarity with core cloud data services: Glue, Redshift, S3, Lambda, EMR, Athena, Fabric, Databricks.
• Knowledge of Spark, Hadoop, Kafka.
• Proficiency in Python and SQL.
• Experience with relational databases: RDS, MS SQL, Azure SQL DB, Postgres.
• IaC: CloudFormation, Terraform, Azure ARM/Bicep.
• CI/CD tooling experience.
• Strong communication and analytical skills.

Responsibilities

• Design and implement scalable data pipelines using Cloud services such as Glue, Redshift, S3, Lambda, EMR, Athena, Microsoft Fabric, andDatabricks.
• Develop and maintain ETL processes to transform and integrate data fromdiverse sources.
• Collaborate with data scientists, analysts, and other stakeholders to understand data requirements and deliver top-notch data solutions.
• Optimize and fine-tune the performance of data pipelines and queries.
• Ensure data quality and integrity through rigorous testing andvalidation processes.
• Implement data security and compliance best practices.
• Monitor and troubleshoot data pipeline issues, ensuring timelyresolutions.
• Stay abreast of the latest developments in data engineeringtechnologies and best practices.

Skills

Data engineering

Python (Pandas)

SQL

Cloud platforms

Big data

Data modeling

CI/CD

IaC

Education

Bachelor's degree

Tools

Databricks

AWS CloudFormation

Terraform

Azure ARM/Bicep

Glue

Redshift

S3

Job description

Calibrate People is seeking a Data Engineer to join our Technology Team in a hybrid Johannesburg setting. The role focuses on building scalable data pipelines and ETL workflows across cloud and on-prem systems, collaborating with data scientists and analysts to deliver actionable insights.

You will leverage tools like AWS/Azure, Spark, and SQL, while ensuring data quality, security, and governance. Strong communication and problem-solving are essential for success in this fast-paced environment.