Semantic Backend Engineer: Pipelines & Embeddings

Infuse; Upington; Remote; ZAR 600 000 - 800 000; Full time

Job summary

A leading technology company in Northern Cape, South Africa, is seeking an applied ML engineer to take ownership of the semantic ingestion pipeline. The role involves developing and optimizing the ETL process, utilizing advanced machine learning models, and ensuring data relevance and freshness. Ideal candidates will have experience in building ML pipelines that benefit real users, along with skills in Python and semantic search technologies.

Qualifications

• Experience in building ML pipelines that reach real users.
• Experience with semantic search and handling unstructured data.
• Familiarity with fresh content identification and indexing.

Responsibilities

• Own the ETL pipeline from raw PDFs to structured resources.
• Implement logic to generate embeddings and perform semantic searches.
• Collaborate with the Tech Lead on UX integration.

Skills

Python

PyTorch

Semantic search

Large-scale tagging

Embedding techniques

Tools

FastAPI

Docker

Milvus

AWS Lambda

Job description

A leading technology company in Northern Cape, South Africa, is seeking an applied ML engineer to take ownership of the semantic ingestion pipeline. The role involves developing and optimizing the ETL process, utilizing advanced machine learning models, and ensuring data relevance and freshness. Ideal candidates will have experience in building ML pipelines that benefit real users, along with skills in Python and semantic search technologies.