Lead Data Engineer: Architect Scalable Pipelines

Future Fit; Johannesburg; Sur place; ZAR 900 000 - 1 300 000; Plein temps

Résumé du poste

Future Fit is seeking a senior Data Engineer to lead the design, development, and maintenance of scalable data pipelines and systems in Johannesburg. You will drive data integration, optimize architecture, and ensure data quality while mentoring junior engineers.

You will collaborate with analysts, scientists, and stakeholders to deliver robust data solutions and stay ahead of industry trends and technologies.

Qualifications

• Bachelor's or Master's degree in Computer Science, Engineering, Information Technology, or a related field.
• Extensive experience as a Data Engineer or in a similar role, with a proven track record of building and maintaining data infrastructure.
• Proficiency in advanced data engineering tools and technologies (e.g., SQL, (Python – AI/ML libraries), Hadoop, Apache Spark, Git and Github, Google Colab, AutoML platforms)
• Strong understanding of database management systems, data warehousing, and big data technologies.
• Experience with cloud platforms (e.g., AWS, Azure, Google Cloud) is highly desirable.
• Excellent problem-solving skills and attention to detail.

Responsabilités

• Lead the design, development, and maintenance of scalable data pipelines and systems.
• Implement and oversee data integration solutions to consolidate data from various sources.
• Optimize data architecture and data flow for performance, reliability, and scalability.
• Collaborate with data analysts, scientists, and other stakeholders to understand data requirements and deliver solutions.
• Ensure data quality and integrity through rigorous testing, validation, and monitoring.
• Develop and maintain ETL processes to extract, transform, and load data.
• Monitor, troubleshoot, and improve data systems to ensure smooth operation.
• Stay updated with the latest industry trends and technologies to continuously enhance data infrastructure.
• Mentor and guide junior data engineers and other team members.

Connaissances

Data system design

Team leadership

Analytical thinking

Independent work

Formation

Bachelor's or Master's degree in CS/Engineering/IT

Outils

SQL

Python (AI/ML)

Hadoop

Apache Spark

Git/GitHub

Google Colab

AutoML platforms

Description du poste

Future Fit is seeking a senior Data Engineer to lead the design, development, and maintenance of scalable data pipelines and systems in Johannesburg. You will drive data integration, optimize architecture, and ensure data quality while mentoring junior engineers.

You will collaborate with analysts, scientists, and stakeholders to deliver robust data solutions and stay ahead of industry trends and technologies.