Senior Data Engineer

Gvw Group, Llc; Wes-Kaap; On-site; ZAR 60 000 - 100 000; Full time

Job summary

An innovative firm is seeking a Senior Data Engineer to join their agile Data Engineering team. This pivotal role involves leading the design and optimization of complex data infrastructures and pipelines, enabling impactful reporting and analytics. You will be responsible for building critical data systems, mentoring junior engineers, and translating business needs into scalable data solutions. With a focus on machine learning and data quality, this position offers the opportunity to make a significant impact in a dynamic environment. If you're passionate about data and eager to drive business success through technology, this is the role for you.

Qualifications

• 5+ years of experience in Python or R for data engineering.
• Strong background in data warehouse architecture and schema design.
• Experience in ML model deployment and frameworks.

Responsibilities

• Develop and maintain scalable data pipelines and API integrations.
• Collaborate with teams to optimize data models for business intelligence.
• Mentor junior engineers and promote best practices.

Skills

Python

R

MS SQL

PostgreSQL

Data Pipeline Development

MLOps

Agile Methodologies

Problem-solving

Communication Skills

Data Quality Monitoring

Education

BSc in Computer Science

MSc in Computer Science

Tools

Power BI

Apache Spark

Kafka

AWS Kinesis

Job description

Aculocity Data Team : Senior Data Engineer

Premise : The Senior Data Engineer is a lead resource responsible for building and developing Aculocity's Data capabilities.

Role : As a Senior Data Engineer, you will be a key member of our agile Data Engineering team, leading the design, development, and optimization of complex data infrastructure and pipelines. Your expertise will enable high-impact reporting, analytics, and machine learning solutions that support business success.

Your responsibilities include building and maintaining critical data systems, mentoring junior engineers, and translating business requirements into scalable data solutions.

Job Title :

Senior Data Engineer

Reports To :

Data Engineering Manager (Functionally)

Data Engineer Job Responsibilities :

1. Serve as the subject matter expert for data and systems.
2. Develop and maintain scalable data pipelines and API integrations to handle increasing data volume and complexity.
3. Collaborate with analytics and business teams to optimize data models for business intelligence and decision-making.
4. Implement data quality monitoring processes to ensure accuracy and availability of production data.
5. Develop end-to-end ML pipelines covering data ingestion, transformation, model training, validation, serving, and evaluation.
6. Work closely with AI scientists to deploy ML algorithms into production.
7. Set up CI/CD/CT pipelines and maintain model repositories for ML models.
8. Deploy models as a service.
9. Contribute to documentation and perform data analysis to troubleshoot issues.
10. Collaborate with cross-functional teams to enhance data models and support analytics.
11. Design data assets, ETL jobs, and data integrations.
12. Establish data quality frameworks and evaluate tools for data lineage.
13. Develop strategies for long-term data platform architecture.
14. Mentor junior engineers, lead code reviews, and promote best practices.

Qualifications / Skills :

• Knowledge of IT operations and best practices for reliable services.
• Experience with data pipeline and model development best practices.
• Familiarity with Agile methodologies.
• Strong problem-solving, troubleshooting, and documentation skills.
• Excellent communication and customer service skills.
• Willingness to learn and stay updated with new technologies.
• Understanding of ML algorithms and experience with MLOps pipelines.
• Collaborative team player mindset.

Education, Experience, and Licensing :

• BSc or MSc in Computer Science or related field.
• Minimum 5 years of experience with Python or R.
• Minimum 5 years of MS SQL experience; PostgreSQL is a plus.
• Experience with data warehouse architecture, schema design, and data modeling.
• Experience with BI tools like Power BI.
• Experience designing and maintaining data processing systems on cloud and on-premises platforms.
• Experience in ML model deployment and frameworks.
• Good experience with Apache Spark.
• Ability to troubleshoot production issues.
• Experience in demos and training for technical and non-technical audiences.

Advantages :

• Experience with data streaming technologies like Kafka or AWS Kinesis.
• Experience with IoT device and systems integration.