Prompt Architect: Reliable, Grounded LLM Systems

Cloud23; Johannesburg; On-site; ZAR 600,000 - 900,000; Full time

Job summary

Cloud23 is seeking a role focused on designing, evaluating and hardening prompting and context strategies for its agentic products. You will ensure outputs are accurate, grounded and repeatable, with an emphasis on measurable improvements.

Key work includes building robust evaluation sets, reducing hallucinations via guardrails and tight integration with retrieval and tooling systems. You will document effective approaches to ensure knowledge persists beyond individuals and teams.

Qualifications

• Experience getting reliable behaviour from large language models in production.
• Ability to measure and validate outcomes rather than assert them.
• Experience working with codebases, not just alongside them.
• Understanding of retrieval, context limits and tool use.

Responsibilities

• Design prompting, context and retrieval strategies for production LLM features.
• Build evaluation sets and regression tests to prove changes are better.
• Reduce hallucination through grounding, constraints and guardrails.
• Collaborate with engineers on retrieval, tools, fallbacks and cost.
• Document what works and why to ensure long-term maintainability.

Skills

Production LLM optimization

Evaluation mindset

Engineering ability

Retrieval and tooling knowledge

Job description

Cloud23 is seeking a role focused on designing, evaluating and hardening prompting and context strategies for its agentic products. You will ensure outputs are accurate, grounded and repeatable, with an emphasis on measurable improvements.

Key work includes building robust evaluation sets, reducing hallucinations via guardrails and tight integration with retrieval and tooling systems. You will document effective approaches to ensure knowledge persists beyond individuals and teams.