Senior Prompt Engineer (AI / LLM)

Ableneo; Wien; Hybrid; EUR 90,000 - 130,000; Full time

Benefits offered by this job

Engaging projects

Learning program

Flexible hours

Home office

Office perks

Job summary

Ableneo is seeking a Senior Prompt Engineer (M/F) for our banking sector client. You will shape and optimize AI agents used in conversational applications, focusing on prompt development, agent configuration and systematic evaluation of LLM-based solutions.

You will design system prompts, configure routing and knowledge sources, and develop tests to ensure accurate, consistent responses while collaborating with engineering and product teams.

Qualifications

• Hands-on experience with prompt engineering for LLM-based apps.
• Experience configuring and optimizing AI agents.
• Strong understanding of system prompts, context management and few-shot prompting.
• Experience with LLM evaluation and systematic testing of AI outputs.
• Knowledge of tool calling, retrieval-augmented generation (RAG) and structured outputs.
• Ability to analyze LLM responses and implement improvements.
• Experience with API requests, tool results and conversation traces.
• Professional working proficiency in English.

Responsibilities

• Design, develop and maintain system prompts, agent instructions and reusable prompt components.
• Configure AI agents, including routing, handover logic, knowledge sources and tool usage.
• Optimize LLM behaviour for accurate, consistent responses.
• Define how AI agents handle ambiguous requests and missing information.
• Design evaluation datasets, test scenarios and scoring criteria.
• Analyze conversation traces and evaluation results to identify issues and improve performance.
• Test and compare prompt and model changes to measure impact on quality.
• Develop regression test cases to ensure consistent behavior over time.
• Collaborate with engineering, product and business teams to translate requirements into reliable AI solutions.
• Document prompt changes, experiments and evaluation results.

Skills

Prompt engineering

LLM configuration

Evaluate outputs

Context management

API requests

Tool calling

Python scripting

English proficiency

Tools

LangChain

Git

JSON schemas

Job description

Job description

We are looking for a Senior Prompt Engineer (M/F) for our client in the banking sector. In this role, you will shape and optimize the behaviour of AI agents used in conversational AI applications. You will focus on prompt development, agent configuration and systematic evaluation of LLM-based solutions to ensure reliable, accurate and consistent outputs.

Responsibilities

• Design, develop and maintain system prompts, agent instructions and reusable prompt components.

• Configure AI agents, including routing, handover logic, knowledge sources and tool usage.

• Optimize LLM behaviour to ensure accurate, consistent and contextually relevant responses.

• Define how AI agents handle ambiguous requests, missing information and situations beyond their capabilities.

• Design evaluation datasets, test scenarios and scoring criteria to measure response quality.

• Analyze conversation traces and evaluation results to identify issues and improve agent performance.

• Test and compare prompt and model changes to measure their impact on output quality.

• Develop regression test cases to ensure consistent agent behaviour over time.

• Collaborate with engineering, product and business teams to translate requirements into reliable AI solutions.

• Document prompt changes, experiments and evaluation results.

Requirements

• Hands-on experience with prompt engineering for LLM-based applications.

• Experience configuring and optimizing AI agents.

• Strong understanding of system prompts, context management and few-shot prompting.

• Experience with LLM evaluation and systematic testing of AI-generated outputs.

• Understanding of tool calling, retrieval-augmented generation (RAG) and structured outputs.

• Ability to analyze LLM responses, identify failure patterns and implement improvements.

• Experience working with API requests, tool results and conversation traces.

• Professional working proficiency in English.

Nice to Have

• Experience with LangChain, LangGraph or similar agent frameworks.

• Knowledge of Python scripting and Git.

• Experience with automated evaluation frameworks and LLM-as-a-Judge approaches.

• Familiarity with JSON schemas and structured data outputs.

• Experience testing multilingual AI applications.

• Previous experience in banking, financial services or other regulated environments.

Benefits

• Engaging projects across diverse industries, offering continuous professional challenges and growth opportunities

• Tailor-made learning & development program, complemented by monthly in-house workshops to boost your skills

• Flexible working hours and the possibility of home office

• Modern, inspiring office environment with daily perks such as breakfast, fresh fruit, and yoga sessions

• MultiSport card to support an active lifestyle and access to a wide range of sports and wellness facilities

Company description

Who we are

Ableneo is a partner for AI and technology-driven transformation. We combine deep software engineering with business consulting to help organisations implement intelligent systems that deliver measurable results and reduce the risk of failed innovation.

What we do

AI & Data Enabling. We deploy chatbots and LLM solutions, automate documents and workflows, and work with Computer Vision. Our focus is on solving concrete business problems with AI and achieving a high success rate in real use cases.

Business Optimisation & Efficiency. We streamline processes, reduce delays, automate operational work, and set up more efficient ways of working.

Product Design & Development. We design and build digital products, customer journeys, and scalable platforms that support long-term growth.

Software Engineering. We offer flexible delivery models. From specialised roles to full teams, supporting both fast-growing companies and established enterprises.

Why join us

We have years of experience and strong references across startups, scale-ups, and large organisations. We work on assignments where AI makes real impact. For example, in the banking sector we validated an LLM solution with a high success rate in a real proof-of-concept.

At Ableneo you will work on projects that push the boundaries of AI and software engineering. You will solve complex problems, collaborate with decision-makers, and help build systems that have immediate business value.

If you want to shape the next generation of intelligent software, let’s talk.