Cloud Data Engineer: Spark, Python & Data Solutions

Boardroom Appointments; Sandton; On-site; ZAR 900,000 - 1,500,000; Full time

Job summary

Boardroom Appointments is seeking a Cloud Data Engineer to build and maintain cloud data solutions. You will work with cross-functional teams to implement new technology and scalable systems that support business goals. Strong Python and Spark experience is required, with a background in data modeling and Ab Initio tools.

The role emphasizes architecture, deployment, and rapid problem solving in an Agile environment, with a focus on high-performance data processing.

Qualifications

• Bachelor's degree in Computer Science or related field required.
• 8 years of experience in a technical role with emphasis on data and algorithm development.
• 8 years of software development experience with strong database and data modeling background.
• 5 years of experience designing and implementing high performance systems.
• Abinitio (FYI - Enterprise Data Platform | Ab Initio).

Responsibilities

• Develops, tests and deploys data extraction, analysis and management solutions.
• Provides direct and responsive support for urgent analytical needs.
• Collaborates with the development team to address project requirements.
• Participates in architecture, design, testing and deployment of applications.
• Uses coding languages, scripting methodologies and tools to solve problems with a custom workflow.
• Performs incremental testing on code, processes and deployments to streamline execution.
• Installs and maintains Spark, Apache Spark, Erlang, Spark cluster.
• Manages day-to-day operations of a Spark environment from provisioning to monitoring.
• Uses Python for debugging and data mining.

Skills

Data engineering

Python

Apache Spark

Erlang

Data modeling

Education

Bachelor's degree in Computer Science or related field

Tools

Abinitio

Apache Spark

Erlang

Python

Job description

Boardroom Appointments is seeking a Cloud Data Engineer to build and maintain cloud data solutions. You will work with cross-functional teams to implement new technology and scalable systems that support business goals. Strong Python and Spark experience is required, with a background in data modeling and Ab Initio tools.

The role emphasizes architecture, deployment, and rapid problem solving in an Agile environment, with a focus on high-performance data processing.