Lead Data Engineer: AI Pipelines & Agentic Systems

Placements24

Gauteng

Remote

ZAR 739,000 - 1,052,000

Full time

34 hours ago
Be an early applicant

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Job summary

Placements24 is seeking a Data Engineer to design and deliver scalable data pipelines, AI-powered workflows, and autonomous systems that interact with APIs, databases, files, and web sources. The role focuses on building reliable production systems with secure engineering practices and robust data quality controls.

You will work on large-scale scraping, LLM integrations, and agentic AI solutions, collaborating with engineers to deploy end-to-end data platforms and infrastructure across modern

Qualifications

  • Strong experience with Python and/or Node.js/TypeScript.
  • Advanced SQL and relational database skills.
  • Experience with AWS or similar cloud platforms.
  • Hands-on experience with data lakes, warehouses, or lakehouse technologies.
  • Practical experience using LLM platforms such as OpenAI, Anthropic, Gemini, Bedrock, or similar.
  • Experience building agentic workflows involving tool use, retrieval systems, memory.
  • Experience with scraping frameworks such as Playwright, Puppeteer, Scrapy, or Selenium.
  • Strong understanding of APIs, containers, CI/CD, testing, and secure engineering practices.

Responsibilities

  • Design and build scalable batch and near-real-time data pipelines.
  • Experience specifically with Claude Code.
  • Ingest and process data from APIs, databases, files, event streams, and web sources.
  • Develop and maintain large-scale scraping and data acquisition systems.
  • Clean, validate, transform, and model data for analytics and AI applications.
  • Build LLM-powered workflows and agentic AI solutions using: Structured outputs, Retrieval-augmented generation (RAG), API and database integrations.
  • Support modern data lake, warehouse, and lakehouse architectures.
  • Implement monitoring, observability, schema validation, and data quality controls.
  • Collaborate with engineering teams to deliver secure, scalable, and maintainable production systems.

Skills

Python
Node.js/TypeScript
SQL
AWS
LLM platforms
APIs
CI/CD
Testing
Security practices

Tools

Playwright
Puppeteer
Scrapy
Selenium
Docker
Kubernetes

Job description

Placements24 is seeking a Data Engineer to design and deliver scalable data pipelines, AI-powered workflows, and autonomous systems that interact with APIs, databases, files, and web sources. The role focuses on building reliable production systems with secure engineering practices and robust data quality controls.

You will work on large-scale scraping, LLM integrations, and agentic AI solutions, collaborating with engineers to deploy end-to-end data platforms and infrastructure across modern

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Remote AI/ML Data Engineer: Scalable Pipelines & Data Quality
Remote AI/ML Data Engineer: Scalable Pipelines & Data Quality

Placements24 • Upington

Hybrid
ZAR 700,000 - 1,200,000
Competitive salary
Health benefits
Paid time off
+3
Remote AI Data Engineer: Scalable Pipelines for ML
Remote AI Data Engineer: Scalable Pipelines for ML

Placements24 • East London

Hybrid
ZAR 700,000 - 1,100,000
Remote-first culture
Health benefits
Professional development
Remote AI Data Engineer — Build Scalable Pipelines
Remote AI Data Engineer — Build Scalable Pipelines

Placements24 • Klerksdorp

Hybrid
ZAR 600,000 - 900,000
Medical aid
Retirement contributions
Remote-first culture
Remote AI Data Engineer: LLM Pipelines & Scraping
Remote AI Data Engineer: LLM Pipelines & Scraping

e-Merge IT Recruitment • South Africa

Remote
ZAR 739,000 - 1,052,000
Data Engineer: AI/LLM & Modern Data Pipelines
Data Engineer: AI/LLM & Modern Data Pipelines

PBT Group • Cape Town

On-site
ZAR 600,000 - 900,000
Remote Lead Data Engineer for AI & Big Data Platforms
Remote Lead Data Engineer for AI & Big Data Platforms

Placements24 • Stellenbosch

Remote
ZAR 1,200,000 - 2,100,000
Top-tier health insurance
Retirement contributions
Home office stipend
+2
Data Engineer: AI Platforms & Scalable Pipelines
Data Engineer: AI Platforms & Scalable Pipelines

Placements24 • Upington

On-site
ZAR 600,000 - 900,000
Competitive salary
Medical aid
Retirement plan
+2
Data Engineer: Build Scalable AI Data Pipelines
Data Engineer: Build Scalable AI Data Pipelines

Greenyard • Cape Town

On-site
ZAR 720,000 - 1,100,000
Lead Data Engineer
Lead Data Engineer

Placements24 • Gauteng

Remote
ZAR 739,000 - 1,052,000
Data Engineer: Pipelines for AI & Big Data Analytics
Data Engineer: Pipelines for AI & Big Data Analytics

Spatialedge • South Africa

On-site
ZAR 600,000 - 900,000