Remote AI Prompt Architect & Evaluation Engineer

Yonder Media Mobile Inc.; Poland; Remote; USD 120,000 - 180,000; Full time

Benefits offered by this job

Flexible working hours

Corporate equipment for work

Competitive salary

Real opportunity for personal and.prof

Job summary

House of YO is seeking a prompt and evaluation specialist for YOlanda, the AI concierge. You will own the system prompts, ensure smooth handoffs to the compute cooperative, and build evaluation sets with measurable scores.

You will also develop data and training pipelines, test robustness against adversarial prompts, and maintain versioned prompts in git. The role is remote, global, full-time, requiring strong English writing and bilingual capabilities, plus Python and LangChain/ LangGraph

Qualifications

• 2+ years building with large language models in production.
• Proficient Python for shipping scripts, API clients, data handling, tests.
• Experience with LangChain and LangGraph; ability to read and modify a graph you didn’t write.
• Experience with retrieval in production: chunking, embeddings, reranking, grounding.
• Exceptional written English and bilingual Spanish and English.
• Methodology: iteratively test one variable, record results, and move forward.

Responsibilities

• Own YOlanda's system prompt — persona, tone, safety rules, refusals — and prompts behind every user action.
• Decide what stays in YOlanda's frame of reference and what is handed off to YOnC, ensuring a seamless handoff.
• Build a few-shot library and manage context so the model doesn’t start from a blank prompt or repeat answered questions.
• Create an evaluation set with tasks, pass criteria and release-traceable scores.
• Rank and critique outputs (RLHF), rewrite weak answers into fine-tuning data, and fact-check plan prices, balances and YOYO$ maths.
• Run tests against adversarial prompts, jailbreak attempts, and abuse of top-up/rewards flows.
• Write Python to run and score evaluations at volume, work with the API (streaming, function calling, structured output), and version prompts in git.

Skills

LLMs in production

Python

LangChain

LangGraph

Retrieval in production

English writing

English/Spanish bilingual

Git versioning

Tools

Git

Job description

House of YO is seeking a prompt and evaluation specialist for YOlanda, the AI concierge. You will own the system prompts, ensure smooth handoffs to the compute cooperative, and build evaluation sets with measurable scores.

You will also develop data and training pipelines, test robustness against adversarial prompts, and maintain versioned prompts in git. The role is remote, global, full-time, requiring strong English writing and bilingual capabilities, plus Python and LangChain/ LangGraph