Prompt Engineer (Healthcare)

Talkie sp. z o.o.

United States

Remote

USD 120,000 - 180,000

Full time

4 days ago

Be an early applicant

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Benefits offered by this job

Private medical care

Multisport card

Annual offsite

Training budget

Remote/hybrid work options

Mac laptop setup

Job summary

Talkie sp. z o.o. is seeking a Prompt Engineer to own the quality and intelligence of our AI Agents across voice, chat, and SMS for US healthcare clients.

You will design, test, and iterate prompts, analyze real patient conversations, and implement tool-use patterns and context caching to improve reliability.

This role demands rigorous QA, clear documentation, and collaboration with product, engineering, and client teams; remote-friendly with CET overlap.

Qualifications

Experience writing production prompts and iterating with live users.
Ability to analyze conversations and identify failure patterns.
Experience with function calling and tool use in LLMs.
Experience with evaluation frameworks and QA for conversational AI.
Excellent written and spoken English (C1+).

Responsibilities

Design, write, and continuously optimise prompts that power our AI Agents — making them natural, accurate, and reliable.
Analyse real patient–agent conversations end-to-end, identifying failure patterns, edge cases, and opportunities to improve agent behaviour.
Propose and implement technical solutions around function calling, tool use, context caching, and other LLM capabilities that make our agents smarter.
Build and run evaluation frameworks to test agent performance before and after changes — because every conversation is with a real patient and there is zero margin for error.
Create clear, structured documentation and customisation instructions so that agents can be tailored to each client’s specific workflows and needs.

Skills

LLM prompts

Analytical rigour

English proficiency

Proactive problem solving

Cross-functional collaboration

Healthcare/regulated domain

Tools

Langfuse

LangSmith

Eval frameworks

API integrations

Job description

Описание:

Talkie builds AI Agents for healthcare. Our agents handle patient–clinic communication end-to-end — voice calls, web chat, and SMS — so patients get 24/7 access to care and busy practices never miss a conversation. Every month, our AI Agents handle close to a million real patient conversations across the US and Poland. Trusted by primary care, specialty practices, and hospitals. We're looking for a Prompt Engineer to own the quality and intelligence of our AI Agents — from prompt design to production. This is a high-impact role at the intersection of language, technology, and healthcare. Your work will directly shape how hundreds of thousands of patients experience care. This is not a research role. You'll be writing, testing, and iterating prompts that run live with real patients — so rigour, empathy, and a zero-error mindset matter as much as technical skill.

Задачи:

Design, write, and continuously optimise prompts that power our AI Agents — making them natural, accurate, and reliable.

Analyse real patient–agent conversations end-to-end, identifying failure patterns, edge cases, and opportunities to improve agent behaviour.

Propose and implement technical solutions around function calling, tool use, context caching, and other LLM capabilities that make our agents smarter.

Build and run evaluation frameworks to test agent performance before and after changes — because every conversation is with a real patient and there is zero margin for error.

Create clear, structured documentation and customisation instructions so that agents can be tailored to each client’s specific workflows and needs.

Stay on top of the rapidly evolving LLM landscape — new models, techniques, and conventions — and bring the best ideas back to the team.

Work closely with the Product Manager (US market), engineering, and client teams to ensure agent quality across all deployments.

What will you achieve with us?

Shape the experience that hundreds of thousands of patients have when they reach out to their doctor — across every channel — and make it better every single day.
Push the boundaries of what LLM-powered AI agents can do in a highly regulated, real-world, multi-channel environment.
Build evaluation and quality systems for conversational AI that don’t exist yet — you’ll be creating the playbook.
Have a direct, measurable impact on patient access to healthcare in both the US.

Требования:

LLM Experience — Hands‑on experience writing and iterating on prompts for production systems. More importantly, you learn fast — this field changes weekly and you keep up.
Analytical Rigour — Ability to review conversations, extract failure patterns, and turn findings into concrete, measurable improvements.
Communication & Collaboration — Comfort on client calls and working across technical and non‑technical teams; you translate clearly in both directions.
This is not a “sit in a cave and prompt all day” role ;). Proactivity — You spot problems before being asked, flag them, and come with a proposed fix.
Zero-Error Mindset — Our agents talk to real patients. You understand the responsibility and bring the precision and care it demands.
English — C1+ English, strong written communication. Our agents talk to US patients.
Shifted Hours — You are available to work 12:00–20:00 CET at least 3 days per week (optimally 5) to overlap with US Eastern Time business hours.
Function Calling & Tool Use - Experience with LLM tool‑use patterns, structured outputs, and API integrations.
Evaluation Frameworks- Familiarity with eval tools — Braintrust, DeepEval, LangSmith, or custom pipelines.
Multi-Channel Experience - Understanding of voice AI nuances: latency, turn-taking, TTS/ASR — and how they differ from chat or SMS.
Healthcare Background - Prior work in healthcare, health‑tech, or regulated industries where accuracy and compliance are non‑negotiable.
Genuine Curiosity - You read release notes. You experiment with new models. You show up on Monday with fresh ideas.

Your goals as a Prompt Engineer

Your goals as a Prompt Engineer Short term — first 3 months Develop a deep understanding of our AI Agent architecture, prompt patterns, client configurations, for our US market product. Audit existing agent conversations, identify the top quality issues, and implement prompt improvements with measurable impact. Take ownership of the agent testing and evaluation process — establish baselines and a repeatable QA workflow. Get up to speed on our tooling (Langfuse, ClickUp, internal platforms) and the team’s ways of working. Longer term — first 12 months Own the end-to‑end prompt and agent quality lifecycle across our US deployments. Build and maintain a structured evaluation framework that catches regressions before they reach patients. Develop comprehensive customisation documentation that enables scalable client onboarding. Become the team’s go‑to expert on LLM capabilities, staying ahead of model releases and new techniques. Contribute to shaping our product roadmap with insights from conversation analysis and agent performance data.

What are we like as a company?

We are friendly, direct, and driven by curiosity and ambition. We value a growth mindset and see failure as a learning opportunity. Our culture is built on inquiry and critical thinking — asking questions is encouraged and thorough investigation is standard. We’re proactive problem-solvers who don’t shy away from challenges. When something’s broken, we acknowledge it, propose solutions, and fix it. And yes — we love to have fun too. Dancing till early hours, karaoke nights... we’ve got a long tradition of good times at Talkie!

Условия:

Competitive pay with benefits: employment contract or B2B contract.