Senior Ai Data Engineer - Hybrid Pipelines For Ml

Placements24; Gauteng; Hybrid; ZAR 972,000 - 1,188,000; Full time

Benefits offered by this job

Hybrid work model

Competitive salary

Career growth opportunities

Job summary

Placements24 is seeking an experienced Data Engineer to design and maintain ETL pipelines and data warehouse loads in a hybrid work environment. You will implement tenant-specific data isolation and collaborate with analytics teams to deliver robust data solutions.

The role requires strong Python and SQL skills, experience with big data tools, and familiarity with cloud data services across AWS/Azure/GCP. Regulatory awareness and data governance are important in this position.

Qualifications

• Bachelor-level degree or higher in Computer Science, IT, or related field.
• 3+ years in data engineering or similar role.
• Experience with big data tools and ETL/ELT pipelines.

Responsibilities

• Design, build, and maintain ETL pipelines for multi-tenant data platform.
• Develop Data Platform APIs for data ingestion, processing and access.
• Ensure data isolation and governance across tenants.
• Maintain data loads into data warehouse and monitor pipelines.

Skills

Python

SQL

Data Warehousing

Cloud data concepts

Education

Bachelor/Degree in CS/IT/Data Eng

Tools

Spark

Hadoop

Azure Data Factory

Cloud platforms (AWS/Azure/GCP)

Job description

An established insurance company is seeking to hire a highly skilled and experienced Data Engineer to join their team. Your:

Formal Education

• Degree in Data Science, Information Technology, Computer Science or equivalent

Advantageous

• Cloud Data Certifications
• Exposure to regulated environments, financial services, fintech

Experience

• Minimum of 2 years in a data engineer role or a similar technical role.

Responsibilities

• Build and maintain ETL pipelines supporting a multi-tenant data platform, ingesting data from APIs, databases, and event sources.
• Build and maintain Data Platform APIs that allow teams to ingest, process, and access data easily and reliably.
• Implemented tenant-specific logic by following existing configuration and naming conventions.
• Apply tenant-level data isolation using schemas, partitions, or access controls
• Build models from existing templates used for financial and operational reporting. Develop models for analytics and reporting, maintaining consistency with shared data models.
• Monitor scheduled pipelines, investigate failures, and resolve data quality issues and inconsistencies.
• Maintain daily and incremental data loads into the data warehouse.
• Assist with onboarding new clients by validating source data and testing pipeline outputs.
• Work closely with senior data engineers to learn patterns for multi-tenant data isolation.
• Collaborate with analytics, product, and customer facing teams to understand reporting needs.
• Support strict regulatory and audit requirements by following data handling, retention, and audit guidelines.
• Handle financial and sensitive data (PII) according to company policies and regulatory standards (e.g. POPIA)
• Apply least-privilege access and role‑based access controls, and support data protection through masking, encryption, and established security standards.

Technical Skills

• Programming languages Good knowledge of programming languages such as Python, especially used for pipeline and data manipulation.
• SQL working experience using SQL for data cleaning, aggregation, data transformation and integration.
• Data Warehouse - have a fundamental understanding of data warehousing solutions and platforms.
• Databases hands on experience working with relational and non-relational databases.
• Cloud computing comfortable building data solutions using cloud hosted services or data platforms.
• Analytics skills strong problem solving skills, understand data characteristics, identify patterns, and data quality issues.
• Data modelling and ETL Is able to communicate and translate business requirements into existing data models.
• Data Pipeline Development build and validate smaller scale data pipelines independently.
• CI/CD and Version Control apply best practice for managing pipelines and data workflows.

Our client is seeking a meticulous and experienced AI Data Engineer to join their team in Soweto. This role is critical for building and maintaining the robust data infrastructure that powers our AI and machine learning initiatives. You will be responsible for designing, developing, and optimizing data pipelines, ensuring data quality, availability, and accessibility for data scientists and ML engineers. The ideal candidate possesses a strong background in data engineering principles, experience with big data technologies, and a passion for enabling data-driven innovation through reliable and efficient data systems.

Key Responsibilities

• Design, build, and maintain scalable and reliable data pipelines for AI/ML workloads.
• Develop ETL processes to ingest, transform, and load data from various sources.
• Ensure data quality, integrity, and accessibility for data scientists and analysts.
• Implement data governance policies and best practices.
• Collaborate with data scientists and ML engineers to understand data requirements and deliver robust data solutions.

Requirements

• Bachelor's degree in Computer Science, Engineering, or a related quantitative field.
• 3+ years of experience in data engineering, with a focus on big data technologies.
• Proficiency in programming languages such as Python, SQL, or Scala.
• Experience with big data platforms like Spark, Hadoop, or similar.
• Familiarity with cloud data services (AWS, Azure, GCP) and data warehousing concepts.
• Strong understanding of data modeling, database design, and data architecture.

Benefits

• Competitive salary and comprehensive benefits package.
• Opportunities for professional growth and training in AI and data technologies.
• Access to modern data infrastructure and cutting-edge tools.
• Hybrid work model offering flexibility.
• Collaborative and innovative work environment.

What You'll Be Doing

• Perform exploratory data analysis (EDA) and validate datasets.
• Use Python extensively for data analysis, investigation and problem-solving.
• Work with real-world and imperfect datasets to identify patterns, issues and insights.
• Support analytics, reporting and insight-driven initiatives.
• Translate client and business questions into clear data outputs and findings.
• Apply data engineering best practices to ensure data is reliable and fit for analytical use.
• Investigate data and communicate findings clearly to both technical and business stakeholders.
• Engage with stakeholders to understand analytical requirements and provide data-driven solutions.

What We're Looking For

• 2+ years' experience in a Data Engineering, Analytics Engineering or similar data-focused role.
• A solid foundation in data engineering concepts.
• Strong hands-on experience working with data for analysis and insight generation.
• Strong Python skills, particularly pandas and NumPy.
• Experience performing data exploration, investigation and validation.
• Comfortable working with complex, imperfect or inconsistent datasets.
• Strong problem‑solving abilities and an analytical mindset.
• The ability to interpret data within a business and client context.
• Strong communication skills, with the ability to explain data findings clearly.

Education

• BSc, BCom, Diploma or equivalent qualification in Computer Science, Information Technology, Data Engineering or a related field will be advantageous.
• Relevant cloud or data certifications will be advantageous.

Experience

• 5+ years of data engineering experience.
• Strong experience with SQL.
• Strong experience with Python.
• Experience developing ETL/ELT pipelines.
• Experience working with cloud data platforms.
• Experience working with large datasets.

Skills

• Python
• SQL
• ETL / ELT.
• Data Warehousing.
• AWS and/or Azure.
• Data Modelling.
• Azure Data Factory.
• Apache Spark.

Location: Gauteng

The reference number for this position is NG60844 which is a permanent, Hybrid role offering a salary of up to R1.08mil per hour salary negotiable based on experience. E‑mail Nokuthula on e‑Merge.co.za or call her for a chat on to discuss this and other opportunities.