Remote AI Data Engineer: LLM Pipelines & Scraping

E-Merge; South Africa; Remote; ZAR 739,000 - 1,052,000; Full time

Job summary

E-Merge seeks a capable Data Engineer to design scalable data pipelines, build AI-powered workflows, and develop autonomous systems interfacing with tools, APIs, databases, and enterprise data sources. You will implement production-grade scraping, data acquisition, and data quality controls in collaboration with engineering teams.

The role emphasizes hands-on building of pipelines and AI-enabled automation, with production-ready architectures and secure engineering practices.

Qualifications

• Strong experience with Python and/or Node.js/TypeScript.
• Advanced SQL and relational database skills.
• Proven experience building production-grade data pipelines.
• Experience with AWS or similar cloud platforms.
• Hands-on experience with data lakes, warehouses, or lakehouse technologies.
• Practical experience using LLM platforms such as OpenAI, Anthropic, Gemini, Bedrock, or similar.
• Experience building agentic workflows involving tool use, task decomposition, retrieval systems, memory, and human-in-the-loop workflows.
• Experience with scraping frameworks such as Playwright, Puppeteer, Scrapy, or Selenium.
• Strong understanding of APIs, containers, CI/CD, testing, and secure engineering practices.

Responsibilities

• Design and build scalable batch and near-real-time data pipelines.
• Ingest and process data from APIs, databases, files, event streams, and web sources.
• Develop and maintain large-scale scraping and data acquisition systems.
• Build LLM-powered workflows and agentic AI solutions using tool calling, structured outputs, and RAG.
• Support modern data lake, warehouse, and lakehouse architectures.
• Implement monitoring, observability, schema validation, and data quality controls.
• Collaborate with engineering teams to deliver secure, scalable production systems.

Skills

Python

Node.js/TypeScript

SQL

AWS

LLM platforms

Agentic workflows

APIs

Containers

CI/CD

Tools

Playwright

Puppeteer

Scrapy

Selenium

Job description

E-Merge seeks a capable Data Engineer to design scalable data pipelines, build AI-powered workflows, and develop autonomous systems interfacing with tools, APIs, databases, and enterprise data sources. You will implement production-grade scraping, data acquisition, and data quality controls in collaboration with engineering teams.

The role emphasizes hands-on building of pipelines and AI-enabled automation, with production-ready architectures and secure engineering practices.