Software Developer/Data Engineer

Software Developer/Data Engineer page is loaded

Software Developer/Data Engineer

Apply locations Altron Campus Cape Town (Plattekloof) time type Full time posted on Posted 11 Days Ago time left to apply End Date: April 28, 2025 (12 days left to apply) job requisition id R-9807

Title

Software Developer/Data Engineer

Job Description

We are looking for a strong Data Engineer to join our MAC (Margin Assurance Cloud) project. MAC is a bespoke in-house developed AWS Serverless Cloud Native application.

Its core functionalities are:

• Data
  • Extract, load and transform data using familiar scripting languages like SQL & Python to produce automated, near real-time Assurance Controls for the MA Team.
• Self-Service
  • Access to Athena using workgroups & federated roles for end users (MA Team) to run their own queries for investigations.
• Reporting & Alerting
  • Capability to create reports from Athena views and send via email to the business customer.
  • Alerting capabilities & Monitoring on File Ingestion.

Main Components (AWS Services used): S3, Athena, Lambda Functions, GLUE, EC2

Main Coding Languages: Python, PySpark, and SQL

In support of our Clients IT strategy, we are progressing with a major migration project of the current on-prem Assurance system utilized by business Margin Assurance residing in FINOPS to the MAC Cloud environment.

We need a passionate and skilled Data Engineer to help the team drive the project to fruition.

The scope of the project build includes:

• Data Ingestion and Transformation
• S3 Loader Output Build
• Athena SOX Validation Reports Build
• Athena SOX Aggregation Reports Build

Delivery will be an agile project managed in monthly sprints. Altron will deliver the requirements as listed and prioritized in the backlog by the Vodacom Technical Delivery Lead and agreed to in each Sprint Planning Ceremony.

KEY RESPONSIBILITIES:

• Implement scalable data pipelines and architectures using technologies like PySpark/Python/SQL.
• Build out distributed data pipelines and compute tier that operates on AWS Lambda and Glue.
• Serve as a technical resource for team members and mentor junior engineers.
• Collaborate with the team to deliver high-quality solutions that meet business requirements.
• Ensure that code is well-designed, maintainable, and adheres to best practices and standards.
• Play a key role in shaping the direction of engineering practices through working on a scrum-size team empowered to organize and ensure sprint deliverables are met as committed during sprint planning ceremonies.
• Follow project development tools like JIRA, Confluence, and GIT.
• Assist DEVOPS Engineer in automation CICD practices.
• Evaluate and recommend new technologies and approaches to improve the performance, scalability, and reliability of our software systems.
• Code complex transformations using loader specifications provided by the BA.
• Work with Big Data sets, knowledgeable in understanding and solving data problems.
• Automate ingestion through building ingestion pipelines using Lambda and Glue.
• Skilled in coding languages PySpark, Python, and SQL.
• Experience with AWS, focusing on the following services: S3, Athena, Lambda Functions, GLUE, EC2.
• Experience in automation with CloudFormation.
• Experience with GIT.

Skills / Requirements:

• 4 to 8 years of Data Engineering or Software Development experience working on Data Driven ecosystems.
• Experience in coding complex transformations using loader specifications provided by the BA.
• Ability to work with Big Data sets, knowledgeable in understanding and solving data problems.
• Experience in automating ingestion through building ingestion pipelines using Lambda or Glue.
• Skilled in coding languages PySpark/Python/SQL.
• Experience with AWS, focusing on the following services: S3, Athena, Lambda Functions, GLUE, EC2.
• AWS certification required.
• Experience with data modeling and data architecture design required.

Educational Qualifications:

• BSc Comp Sci/BEng

Professional Qualifications:

• AWS Professional Certification

Years of Experience:

• 4 to 8 years of Data Engineering or Software Development experience working on Data Driven ecosystems.

Education:

Bachelor's Degree: Computer and Information Science (Required)

Languages:

English