Senior Data Engineer

Future Fit; Johannesburg; On-site; ZAR 900,000 - 1,300,000; Full time

Job summary

Future Fit is seeking a senior Data Engineer to lead the design, development, and maintenance of scalable data pipelines and systems in Johannesburg. You will drive data integration, optimize architecture, and ensure data quality while mentoring junior engineers.

You will collaborate with analysts, scientists, and stakeholders to deliver robust data solutions and stay ahead of industry trends and technologies.

Qualifications

• Bachelor's or Master's degree in Computer Science, Engineering, Information Technology, or a related field.
• Extensive experience as a Data Engineer or in a similar role, with a proven track record of building and maintaining data infrastructure.
• Proficiency in advanced data engineering tools and technologies (e.g., SQL, (Python – AI/ML libraries), Hadoop, Apache Spark, Git and Github, Google Colab, AutoML platforms)
• Strong understanding of database management systems, data warehousing, and big data technologies.
• Experience with cloud platforms (e.g., AWS, Azure, Google Cloud) is highly desirable.
• Excellent problem-solving skills and attention to detail.

Responsibilities

• Lead the design, development, and maintenance of scalable data pipelines and systems.
• Implement and oversee data integration solutions to consolidate data from various sources.
• Optimize data architecture and data flow for performance, reliability, and scalability.
• Collaborate with data analysts, scientists, and other stakeholders to understand data requirements and deliver solutions.
• Ensure data quality and integrity through rigorous testing, validation, and monitoring.
• Develop and maintain ETL processes to extract, transform, and load data.
• Monitor, troubleshoot, and improve data systems to ensure smooth operation.
• Stay updated with the latest industry trends and technologies to continuously enhance data infrastructure.
• Mentor and guide junior data engineers and other team members.

Skills

Data system design

Team leadership

Analytical thinking

Independent work

Education

Bachelor's or Master's degree in CS/Engineering/IT

Tools

SQL

Python (AI/ML)

Hadoop

Apache Spark

Git/GitHub

Google Colab

AutoML platforms

Job description

Reports To: [Head of Data Science / Director of Analytics / CTO]

Key Responsibilities

• Lead the design, development, and maintenance of scalable data pipelines and systems.
• Implement and oversee data integration solutions to consolidate data from various sources.
• Optimize data architecture and data flow for performance, reliability, and scalability.
• Collaborate with data analysts, scientists, and other stakeholders to understand data requirements and deliver solutions.
• Ensure data quality and integrity through rigorous testing, validation, and monitoring.
• Develop and maintain ETL processes to extract, transform, and load data.
• Monitor, troubleshoot, and improve data systems to ensure smooth operation.
• Stay updated with the latest industry trends and technologies to continuously enhance data infrastructure.
• Mentor and guide junior data engineers and other team members.

Qualifications

• Bachelors or Masters degree in Computer Science, Engineering, Information Technology, or a related field.
• Extensive experience as a Data Engineer or in a similar role, with a proven track record of building and maintaining data infrastructure.
• Proficiency in advanced data engineering tools and technologies (e.g., SQL, (Python – AI/ML libraries), Hadoop, Apache Spark, Git and Github, Google Colab, AutoML platforms)
• Strong understanding of database management systems, data warehousing, and big data technologies.
• Experience with cloud platforms (e.g., AWS, Azure, Google Cloud) is highly desirable.
• Excellent problem-solving skills and attention to detail.

Skills

• Ability to design and implement complex data systems.
• Proficiency in programming languages such as Python, Java, or Scala.
• Experience with big data technologies and frameworks.
• Knowledge of data modelling and database design.
• Ability to work independently and lead a team.
• Strong analytical and troubleshooting skills.
• Leadership and mentoring abilities.
• Opportunities for professional development and growth.
• Leadership opportunities within the data engineering team