Prompt Engineer

Cloud23; Johannesburg; On-site; ZAR 600,000 - 900,000; Full time

Job summary

Cloud23 is seeking a role focused on designing, evaluating and hardening prompting and context strategies for its agentic products. You will ensure outputs are accurate, grounded and repeatable, with an emphasis on measurable improvements.

Key work includes building robust evaluation sets, reducing hallucinations via guardrails and tight integration with retrieval and tooling systems. You will document effective approaches to ensure knowledge persists beyond individuals and teams.

Qualifications

• Experience getting reliable behaviour from large language models in production.
• Ability to measure and validate outcomes rather than assert them.
• Experience working with codebases, not just alongside them.
• Understanding of retrieval, context limits and tool use.

Responsibilities

• Design prompting, context and retrieval strategies for production LLM features.
• Build evaluation sets and regression tests to prove changes are better.
• Reduce hallucination through grounding, constraints and guardrails.
• Collaborate with engineers on retrieval, tools, fallbacks and cost.
• Document what works and why to ensure long-term maintainability.

Skills

Production LLM optimization

Evaluation mindset

Engineering ability

Retrieval and tooling knowledge

Job description

Design, evaluate and harden the prompting and context strategies behind Cloud23's agentic products, so outputs are accurate, grounded and repeatable.

What you'll do

• Design prompting, context and retrieval strategies for production LLM features.
• Build evaluation sets and regression tests so a change can be proven better, not just felt better.
• Reduce hallucination through grounding, constraints and guardrails rather than hope.
• Work with engineers on the surrounding system: retrieval, tools, fallback and cost.
• Document what works and why, so it survives you.

What we're looking for

• Demonstrable experience getting reliable behaviour from large language models in production.
• An evaluation mindset - you measure rather than assert.
• Enough engineering ability to work inside the codebase, not alongside it.
• Understanding of retrieval, context limits and tool use.
• Experience with governed or regulated deployments is valued.