Data Engineer Midrand

Cruze Control Technologies; Gauteng; On-site; ZAR 200 000 - 300 000; Full time

Job summary

A technology solutions company located in South Africa, Gauteng is seeking an AWS Data Engineer. The role includes building and maintaining Big Data Pipelines and ensuring data accuracy through thorough testing. Candidates should have 3–5 years of experience in data engineering with a strong background in AWS services, Python, and ETL processes. Relevant certifications and prior automotive industry experience are preferred.

Qualifications

• 3 – 5 years working experience required.
• Experience developing and working with REST APIs is a bonus.
• Basic experience in Networking and troubleshooting network issues.

Responsibilities

• Build and maintain Big Data Pipelines using Data Platforms.
• Perform thorough testing and data validation for accuracy.
• Prepare specifications for program development.

Skills

Analytical skills

Data validation

Communication skills

Team player

AWS Glue

AWS S3

AWS RDS

Python 3.x

Docker

ETL

Education

Degree

AWS Certified Cloud Practitioner

AWS Certified SysOps Associate

AWS Certified Developer Associate

AWS Certified Architect Associate

AWS Certified Architect Professional

Hashicorp Certified Terraform Associate

Tools

Confluence

JIRA

Great Expectations

Terraform

PySpark

Boto3

Powershell

Bash

Job description

AWS Data EngineerThe AWS Data Engineer will be responsible for building and maintaining Big Data Pipelines using Data Platforms and must ensure that data is shared in line with the information classification requirements on a need-to-know basis.Duties & ResponsibilitiesExceptional analytical skills analyzing large and complex data sets.Perform thorough testing and data validation to ensure the accuracy of data transformations.Strong written and verbal communication skills, with precise documentation.Self-driven team player with ability to work independently and multi-task.Experience building data pipeline using AWS Glue or Data Pipeline, or similar platforms.Familiar with data store such as AWS S3, and AWS RDS or DynamoDB.Experience and solid understanding of various software design patterns.Experience preparing specifications from which programs will be written, designed, coded, tested and debugged.Strong organizational skills.

Experience developing and working with REST APIs is a bonus.Basic experience in Networking and troubleshooting network issues.Desired Experience & QualificationDegreeCertifications such as AWS Certified Cloud Practitioner, AWS Certified SysOps Associate, AWS Certified Developer Associate, AWS Certified Architect Associate, AWS Certified Architect Professional, Hashicorp Certified Terraform Associate.Automotive industry experience.Terraform.Python 3.x.PySpark.Boto3.ETL.Docker.Powershell / Bash.Must have 3 – 5 years working experience.3 – 5 years experience in working with Enterprise Collaboration tools such as Confluence, JIRA.3 – 5 years experience developing technical documentation and artifacts.Knowledge of data formats such as Parquet, AVRO, JSON, XML, CSV.Experience working with Data Quality Tools such as Great Expectations.Knowledge of the Agile Working Model.Market related.