Senior Data Engineer

Indsafri; South Africa; On-site; ZAR 900,000 - 1,500,000; Full time

Job summary

Indsafri is seeking a Senior Data Engineer to design, build, and maintain scalable data pipelines and platforms. You will drive technical excellence in an Agile team, ensuring high data availability while adopting AI-driven engineering practices within a governance-focused framework.

You will mentor junior engineers, enforce CI/CD and documentation standards, and work to enable trusted analytics across the organization.

Qualifications

• Bachelor's degree in CS/IS/Engineering or related field.
• 4–7+ years of data engineering experience.
• Advanced Python and SQL
• Experience with AWS-based data architectures
• Strong data modeling, schema design, and data warehousing knowledge
• Experience ingesting data from relational DBs, APIs, and cloud storage
• Git, CI/CD pipelines, and automation experience
• Experience integrating AI tools in engineering workflows with risk management
• Strong analytical, problem-solving, and stakeholder communication skills

Responsibilities

• Lead design, development, and operational support of scalable data pipelines and platforms.
• Architect and implement batch and incremental data ingestion and transformation pipelines with monitoring.
• Contribute to evolution of the AWS-based data environment with security and cost optimization in mind.
• Provide technical leadership and mentorship to junior engineers.
• Enforce engineering standards: version control, automated testing, CI/CD, documentation.
• Champion AI-driven engineering while ensuring data security and governance.
• Maintain platform readiness, incident response, and root-cause analysis.
• Integrate privacy, security, and access controls by design for regulatory alignment.

Skills

Python

SQL

AWS

Data Modeling

Data Warehousing

APIs

Git

CI/CD

AI Tools

Stakeholder Communication

Education

Bachelors in CS/IS/Engineering

Tools

Git

CI/CD

Automation

Job description

We are seeking a highly skilled and proactive Senior Data Engineer to lead the design, development, and operational support of scalable data pipelines and platforms. As a key member of our Agile team, you will be responsible for driving technical excellence, ensuring high-quality data availability, and guiding the team in adopting AI-driven engineering practices within a secure, governance-aligned framework. You will play a pivotal role in enabling trusted analytics and business insights across the organization.

Responsibilities:

• Agile Delivery & Ownership: Lead data engineering outcomes by participating in sprint planning, task decomposition, and estimation, ensuring outputs meet high performance and quality standards.
• Pipeline Development: Architect and implement scalable batch and incremental data ingestion and transformation pipelines with robust error handling and monitoring.
• Platform Strategy: Contribute to the evolution of the AWS-based data environment, ensuring design choices align with organizational security, cost, and performance goals.
• Technical Leadership & Mentorship: Provide guidance to junior engineers, foster a culture of continuous learning, and establish shared engineering standards and best practices.
• Engineering Quality: Enforce rigorous standards including version control, automated testing, CI/CD, and thorough technical documentation.
• AI-Driven Engineering: Champion the responsible and governed use of AI tools to accelerate development, optimize pipeline logic, and improve documentation while ensuring data security and compliance.
• Operational Excellence: Maintain platform readiness, manage incident response, and perform root cause analysis to ensure system stability.
• Governance & Compliance: Integrate privacy, security, and access controls into data solutions by design, ensuring alignment with regulatory and audit requirements.

Required Skills and Qualifications:

• Education: Bachelor’s degree in computer science, Information Systems, Engineering, or a related field.
• Experience: 4-7+ years of professional experience in data engineering or a similar technical role.
• Technical Expertise: Advanced proficiency in Python and SQL.
• Extensive experience with AWS-based data architectures.
• Strong background in data modeling, schema design, and data warehousing principles.
• Experience ingesting data from diverse sources (relational databases, APIs, cloud storage).
• Proficiency in Git, CI/CD pipelines, and automation.
• AI Integration: Proven experience implementing AI tools within engineering workflows, with a strict understanding of managing AI-related risks and validating outputs.
• Soft Skills: Strong analytical problem-solving, effective communication, and the ability to collaborate with both technical and non-technical stakeholders.

Preferred Qualifications:

• Relevant professional certifications (e.g., AWS Certified Data Engineer, Cloud Solutions Architect).
• Experience working within SAFe (Scaled Agile Framework) environments.
• Familiarity with BI and visualization tools, specifically Power BI.
• Strong in AWS, SQL & python experience

Skills

GIT Python AWS SQL Problem-Solving communication Automation CI/CD Data Warehousing Data Modeling Schema Design AI Tools