Prompt Engineer — AI Agents for Real-World Healthcare

Talkie sp. z o.o.; Polska; Hybride; PLN 180 000 - 240 000; Plein temps

Avantages offerts par ce poste

Flexible working

Mac laptop setup

Private medical care

Multisport card

Annual offsite

Training budget

Résumé du poste

Talkie sp. z o.o. is seeking a Prompt Engineer to own the quality and intelligence of our AI Agents in healthcare. You will write, test, and iterate prompts that run live with real patients, requiring rigour, empathy, and a zero-error mindset.

You'll design prompts, analyze conversations, improve agent behaviour, and collaborate with product managers, engineering, and client teams to ensure high-quality deployments across multi-channel channels.

Qualifications

• LLM Experience — Hands-on experience writing and iterating on prompts for production systems.
• Analytical Rigour — Ability to review conversations, extract failure patterns, and turn findings into concrete, measurable improvements.
• Communication & Collaboration — Comfort on client calls and working across technical and non-technical teams; you translate clearly in both directions. This is not a “sit in a cave and prompt all day” role.
• Proactivity — You spot problems before being asked, flag them, and come with a proposed fix.
• Zero-Error Mindset — Our agents talk to real patients. You understand the responsibility and bring the precision and care it demands.
• English — Strong written English for US-based communications.
• Shifted Hours — You are available to work 12:00–20:00 CET at least 3 days per week (optimally 5) to overlap with US Eastern Time business hours.

Responsabilités

• Design, write, and continuously optimise prompts that power our AI Agents — making them natural, accurate, and reliable.
• Analyse real patient–agent conversations end-to-end, identifying failure patterns, edge cases, and opportunities to improve agent behaviour.
• Propose and implement technical solutions around function calling, tool use, context caching, and other LLM capabilities that make our agents smarter.
• Build and run evaluation frameworks to test agent performance before and after changes — because every conversation is with a real patient and there is zero margin for error.
• Create clear, structured documentation and customisation instructions so that agents can be tailored to each client’s specific workflows and needs.
• Stay on top of the rapidly evolving LLM landscape — new models, techniques, and conventions — and bring the best ideas back to the team.
• Work closely with the Product Manager (US market), engineering, and client teams to ensure agent quality across all deployments.

Connaissances

LLM Experience

Analytical Rigour

Communication & Collaboration

Proactivity

Zero-Error Mindset

English Proficiency

Shifted Hours

Outils

Function Calling & Tool Use

Evaluation Frameworks

Multi-Channel Experience

Healthcare Background

Genuine Curiosity

Description du poste

About us

Talkie builds AI Agents for healthcare. Our agents handle patient–clinic communication end-to-end — voice calls, web chat, and SMS — so patients get 24/7 access to care and busy practices never miss a conversation. Every month, our AI Agents handle close to a million real patient conversations across the US and Poland. Trusted by primary care, specialty practices, and hospitals.

The Role

We're looking for a Prompt Engineer to own the quality and intelligence of our AI Agents — from prompt design to production. This is a high-impact role at the intersection of language, technology, and healthcare. Your work will directly shape how hundreds of thousands of patients experience care.

This is not a research role. You'll be writing, testing, and iterating prompts that run live with real patients — so rigour, empathy, and a zero-error mindset matter as much as technical skill.

What You'll Do

• Design, write, and continuously optimise prompts that power our AI Agents — making them natural, accurate, and reliable.

• Analyse real patient–agent conversations end-to-end, identifying failure patterns, edge cases, and opportunities to improve agent behaviour.

• Propose and implement technical solutions around function calling, tool use, context caching, and other LLM capabilities that make our agents smarter.

• Build and run evaluation frameworks to test agent performance before and after changes — because every conversation is with a real patient and there is zero margin for error.

• Create clear, structured documentation and customisation instructions so that agents can be tailored to each client’s specific workflows and needs.

• Stay on top of the rapidly evolving LLM landscape — new models, techniques, and conventions — and bring the best ideas back to the team.

• Work closely with the Product Manager (US market), engineering, and client teams to ensure agent quality across all deployments.

What will you achieve with us?

• Shape the experience that hundreds of thousands of patients have when they reach out to their doctor — across every channel — and make it better every single day.

• Push the boundaries of what LLM-powered AI agents can do in a highly regulated, real-world, multi-channel environment.

• Build evaluation and quality systems for conversational AI that don’t exist yet — you’ll be creating the playbook.

• Have a direct, measurable impact on patient access to healthcare in both the US.

This is a young and fast-moving field. We care less about years of experience and more about how you think, learn, and work.

Must have

• LLM Experience — Hands-on experience writing and iterating on prompts for production systems. More importantly, you learn fast — this field changes weekly and you keep up.

• Analytical Rigour — Ability to review conversations, extract failure patterns, and turn findings into concrete, measurable improvements.

• Communication & Collaboration — Comfort on client calls and working across technical and non-technical teams; you translate clearly in both directions. This is not a “sit in a cave and prompt all day” role ;).

• Proactivity — You spot problems before being asked, flag them, and come with a proposed fix.

• Zero-Error Mindset — Our agents talk to real patients. You understand the responsibility and bring the precision and care it demands.

• English — C1+ English, strong written communication. Our agents talk to US patients.

• Shifted Hours — You are available to work 12:00–20:00 CET at least 3 days per week (optimally 5) to overlap with US Eastern Time business hours.

Nice to have

• Function Calling & Tool Use - Experience with LLM tool-use patterns, structured outputs, and API integrations.

• Evaluation Frameworks- Familiarity with eval tools — Braintrust, DeepEval, LangSmith, or custom pipelines.

• Multi-Channel Experience - Understanding of voice AI nuances: latency, turn-taking, TTS/ASR — and how they differ from chat or SMS.

• Healthcare Background - Prior work in healthcare, health-tech, or regulated industries where accuracy and compliance are non-negotiable.

• Genuine Curiosity - You read release notes. You experiment with new models. You show up on Monday with fresh ideas.

Your goals as a Prompt Engineer

Short term — first 3 months

• Develop a deep understanding of our AI Agent architecture, prompt patterns, client configurations, for our US market product.

• Audit existing agent conversations, identify the top quality issues, and implement prompt improvements with measurable impact.

• Take ownership of the agent testing and evaluation process — establish baselines and a repeatable QA workflow.

• Get up to speed on our tooling (Langfuse, ClickUp, internal platforms) and the team’s ways of working.

Longer term — first 12 months

• Own the end-to-end prompt and agent quality lifecycle across our US deployments.

• Build and maintain a structured evaluation framework that catches regressions before they reach patients.

• Develop comprehensive customisation documentation that enables scalable client onboarding.

• Become the team’s go-to expert on LLM capabilities, staying ahead of model releases and new techniques.

• Contribute to shaping our product roadmap with insights from conversation analysis and agent performance data.
  

What we offer

• Competitive pay with benefits: employment contract or B2B contract.

• A role with real purpose — we’re changing how patients access healthcare in the US.

• Flexible working arrangements — remote, office, or hybrid.

• Work equipment — Mac laptop, monitors, keyboard, mouse, and a setup for both office and home (including a comfy chair).

• Benefits: private medical care, Multisport card, annual offsite, training budget.

• Unique company culture based on mutual trust, honest feedback, and autonomy.

• Working with cutting-edge AI technology on a product that’s genuinely useful to real people.

• A structured onboarding process to help you find your feet.

What are we like as a company?

We are friendly, direct, and driven by curiosity and ambition. We value a growth mindset and see failure as a learning opportunity. Our culture is built on inquiry and critical thinking — asking questions is encouraged and thorough investigation is standard. We’re proactive problem-solvers who don’t shy away from challenges. When something’s broken, we acknowledge it, propose solutions, and fix it. And yes — we love to have fun too. Dancing till early hours, karaoke nights... we’ve got a long tradition of good times at Talkie!

Our recruitment process

Reflecting our culture, our recruitment process is respectful and collaborative. We aim to create a welcoming environment where you can be yourself and get to know us better. The entire process typically takes 2–3 weeks.

What to expect

• Initial Contact — we'll reach out by email or phone if your application is a fit.

• First Interview (1h online) — mutual fit and getting to know each other.

• Second Interview (1.5h online) — practical, hands-on case study/task

• Reference Check — We’ll ask for references and verify them.

• Offer — We’ll make an offer or share detailed feedback.