Prompt & Evaluation Engineer — Remote US-Based

Pop-Up Talent; United States; Hybrid; USD 140,000 - 180,000; Full time

Benefits offered by this job

Health insurance

401K with employer matching

Discretionary Time off

Paid holidays

Job summary

A leading technology recruitment firm is seeking a Prompt & Evaluation Engineer to design and evaluate prompts for AI models while collaborating across teams. The role emphasizes crafting effective prompts, developing evaluation frameworks, and integrating tools for reliability. Ideal candidates will bring 3–5 years of experience in applied AI or NLP, along with strong Python skills. This role offers a mid-senior level position with a full-time employment structure and opportunities for growth.

Qualifications

• 3–5 years in applied AI, NLP, or related software engineering roles.
• Strong Python skills for building eval pipelines.
• Experience designing and iterating prompts for LLMs in production.

Responsibilities

• Design, test, and refine prompts for LLMs.
• Build and maintain evaluation frameworks.
• Collaborate with teams to translate workflows into AI behaviors.

Skills

Applied AI

NLP

Python

Data analysis

LLM evaluation frameworks

Frameworks like LangChain

Strong communication skills

Tools

LangChain

LangGraph

Job description

A leading technology recruitment firm is seeking a Prompt & Evaluation Engineer to design and evaluate prompts for AI models while collaborating across teams. The role emphasizes crafting effective prompts, developing evaluation frameworks, and integrating tools for reliability. Ideal candidates will bring 3–5 years of experience in applied AI or NLP, along with strong Python skills. This role offers a mid-senior level position with a full-time employment structure and opportunities for growth.