Lead Data Engineer

Placements24

Gauteng

Remote

ZAR 739,000 - 1,052,000

Full time

34 hours ago
Be an early applicant

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Job summary

Placements24 is seeking a Data Engineer to design and deliver scalable data pipelines, AI-powered workflows, and autonomous systems that interact with APIs, databases, files, and web sources. The role focuses on building reliable production systems with secure engineering practices and robust data quality controls.

You will work on large-scale scraping, LLM integrations, and agentic AI solutions, collaborating with engineers to deploy end-to-end data platforms and infrastructure across modern

Qualifications

  • Strong experience with Python and/or Node.js/TypeScript.
  • Advanced SQL and relational database skills.
  • Experience with AWS or similar cloud platforms.
  • Hands-on experience with data lakes, warehouses, or lakehouse technologies.
  • Practical experience using LLM platforms such as OpenAI, Anthropic, Gemini, Bedrock, or similar.
  • Experience building agentic workflows involving tool use, retrieval systems, memory.
  • Experience with scraping frameworks such as Playwright, Puppeteer, Scrapy, or Selenium.
  • Strong understanding of APIs, containers, CI/CD, testing, and secure engineering practices.

Responsibilities

  • Design and build scalable batch and near-real-time data pipelines.
  • Experience specifically with Claude Code.
  • Ingest and process data from APIs, databases, files, event streams, and web sources.
  • Develop and maintain large-scale scraping and data acquisition systems.
  • Clean, validate, transform, and model data for analytics and AI applications.
  • Build LLM-powered workflows and agentic AI solutions using: Structured outputs, Retrieval-augmented generation (RAG), API and database integrations.
  • Support modern data lake, warehouse, and lakehouse architectures.
  • Implement monitoring, observability, schema validation, and data quality controls.
  • Collaborate with engineering teams to deliver secure, scalable, and maintainable production systems.

Skills

Python
Node.js/TypeScript
SQL
AWS
LLM platforms
APIs
CI/CD
Testing
Security practices

Tools

Playwright
Puppeteer
Scrapy
Selenium
Docker
Kubernetes

Job description

We are seeking a highly capable Data Engineer with strong experience in scalable data platforms, large-scale scraping, LLM integrations, and agentic AI systems. This role is ideal for a hands‑on builder who enjoys solving complex data challenges and developing intelligent automation solutions that operate reliably in production environments.

You will play a key role in designing and building modern data pipelines, AI-powered workflows, and autonomous systems that can interact with tools, APIs, databases, and enterprise data sources.

Responsibilities
  • Design and build scalable batch and near-real-time data pipelines.
  • Experience specifically with Claude Code.
  • Ingest and process data from APIs, databases, files, event streams, and web sources.
  • Develop and maintain large-scale scraping and data acquisition systems.
  • Clean, validate, transform, and model data for analytics and AI applications.
  • Build LLM-powered workflows and agentic AI solutions using:
  • Structured outputs
  • Retrieval-augmented generation (RAG)
  • API and database integrations
  • Support modern data lake, warehouse, and lakehouse architectures.
  • Implement monitoring, observability, schema validation, and data quality controls.
  • Collaborate with engineering teams to deliver secure, scalable, and maintainable production systems.
Qualifications and experience
  • Strong experience with Python and/or Node.js/TypeScript.
  • Advanced SQL and relational database skills.
  • Experience with AWS or similar cloud platforms.
  • Hands‑on experience with data lakes, warehouses, or lakehouse technologies.
  • Practical experience using LLM platforms such as OpenAI, Anthropic, Gemini, Bedrock, or similar.
  • Experience building agentic workflows involving:
  • Tool use
  • Retrieval systems
  • Memory
  • Experience with scraping frameworks such as Playwright, Puppeteer, Scrapy, or Selenium.
  • Strong understanding of APIs, containers, CI/CD, testing, and secure engineering practices.

The reference number for this position is NG61354 which is a contract remote role offering a salary of up to R650 per hour salary negotiable based on experience.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Lead Data Engineer
Lead Data Engineer

Placements24 • Paarl

On-site
ZAR 1,200,000 - 2,000,000
Competitive salary
Medical aid
Retirement fund
+1
Lead Data Engineer: AI Pipelines & Agentic Systems
Lead Data Engineer: AI Pipelines & Agentic Systems

Placements24 • Gauteng

Remote
ZAR 739,000 - 1,052,000
AI Engineer (Data & Agentic AI)
AI Engineer (Data & Agentic AI)

JVR Software • Johannesburg

On-site
ZAR 559,000 - 669,000
Lead Data Engineer
Lead Data Engineer

Placements24 • Polokwane

Hybrid
ZAR 900,000 - 1,800,000
Medical insurance
Life insurance
Retirement fund
+3
AI Engineer (Claude Code, GitHub Copilot, Cursor) - Remote - up to R800k per annum
AI Engineer (Claude Code, GitHub Copilot, Cursor) - Remote - up to R800k per annum

e-Merge IT Recruitment • South Africa

Remote
ZAR 480,000 - 800,000
AI/ML Data Engineer
AI/ML Data Engineer

Placements24 • Upington

Hybrid
ZAR 480,000 - 720,000
Remote work
AI Data Engineer - Large-Scale Data Pipelines
AI Data Engineer - Large-Scale Data Pipelines

Placements24 • East London

Hybrid
ZAR 700,000 - 1,100,000
Remote-first culture
Health benefits
Professional development
Ai/Ml Data Engineer
Ai/Ml Data Engineer

Placements24 • Johannesburg

On-site
ZAR 600,000 - 900,000
Medical insurance
Dental insurance
Vision insurance
+4
Senior Data Engineer - Big Data & Ai Infrastructure
Senior Data Engineer - Big Data & Ai Infrastructure

Placements24 • Johannesburg

Hybrid
ZAR 900,000 - 1,600,000
Health insurance
Retirement plans
Paid time off
+2
Data Engineer – AI / Agentic AI Exposure
Data Engineer – AI / Agentic AI Exposure

PBT Group • Cape Town

On-site
ZAR 600,000 - 900,000