Data Engineer

Eqplus; Cape Town; On-site; ZAR 500,000 - 800,000; Full time

Job summary

A leading brand in retail is seeking a Data Engineer to design and implement scalable processes for managing large data sets. This role involves direct collaboration with stakeholders, optimizing data pipelines, and employing cutting-edge technologies. It offers the opportunity to have a direct impact on the business and requires a strong commitment to professional development and innovation.

Qualifications

• 4 years of relevant data engineering experience.
• Strong understanding of data structures and algorithms.
• Experience with structured/unstructured data and data management processes.

Responsibilities

• Designing and implementing scalable processes to manage large data sets.
• Developing and maintaining data pipelines and constructing data formats.
• Collaborating with stakeholders to deliver solutions and improving processes.

Skills

Data Structures

Software Design

Python

Scala

Cloud Tools

Big Data

API Development

Analytical Skills

Education

Bachelor's degree in Computer Science

Bachelor's degree in Business Informatics

Bachelor's degree in Mathematics

Bachelor's degree in Engineering

Tools

AWS

Apache Spark

Git

Hadoop

Redshift

Apache Hive

Kafka

Job description

A leading brand in retail is seeking a Data Engineer to assist in designing and implementing scalable processes for managing large data sets. This is an exciting opportunity to work with cuttingedge technology and have a direct impact on the business.

Key Responsibilities :

Designing and implementing scalable processes to manage large data sets ensuring quality and consistency across sources. This involves developing and maintaining data pipelines creating optimal extraction patterns and aligning cloud strategy with data architecture. As the Data Engineer you will also engineer data formats for analytics develop APIs for applications and collaborate with stakeholders to deliver solutions while driving process improvements enhancing scalability and supporting testing and deployment of new features.

Minimum Requirements :

• At least 4 years of relevant data engineering experience
• Bachelors degree in computer science business informatics mathematics statistics engineering or related field
• Strong understanding of data structures algorithms and software design
• Experience with structured and unstructured data data stores RDBMS and data warehouses
• Proficiency in cloud tools (e.g. Redshift Athena Glue Lambda)
• Proficiency in programming languages such as Python Scala Java or C
• Experience with Apache Spark and cloud services (AWS Azure GCP)
• Knowledge of version control systems like Git and SVN
• Familiarity with DevOps practices including CI / CD and Infrastructure as Code
• Exposure to AWS platform environments such as CodePipeline CodeBuild CodeDeploy
• Experience with big data ETL and data management processes
• Experience in API development and cloud architecture
• Strong analytical and problemsolving abilities
• Ability to work collaboratively in an agile environment

Additional Skills & Attributes needed :

• Ability to work effectively with crossfunctional teams
• Strong communication skills to articulate complex data concepts to nontechnical stakeholders
• Continuous learning mindset and commitment to professional development
• Innovative thinking and flexibility to adapt to changing business needs
• Integrity accountability and strong work ethic
• Enthusiasm for experimenting with new technologies and methodologies

Key Skills

Apache Hive,S3,Hadoop,Redshift,Spark,AWS,Apache Pig,NoSQL,Big Data,Data Warehouse,Kafka,Scala

Employment Type : Full-Time

Experience : years

Vacancy : 1

Create a job alert for this search