Expert LLM Engineer (AI Security)

Framna ApS; Gdańsk; Hybrid; PLN 260,000 - 380,000; Full time

Benefits offered by this job

Private medical care (Compensa)

MultiSport

Hybrid in-studio collaboration (Gdańsk

MacBook and testing devices

Training budget

English classes

Job summary

Framna seeks a senior engineer to build and scale AI-powered security solutions for a passwordless authentication platform used by millions. You will leverage Python and LLMs, develop RAG and multi-agent systems, and ensure reliability, performance, and cost efficiency.

You will probe vulnerabilities, design agent tooling, and monitor activity to detect malicious behavior. Hybrid work in Gdańsk with occasional in-person collaboration.

Qualifications

• 7+ years in software, ML, or security engineering with 2+ years in production LLM-based solutions.
• Building maintainable backend components that integrate multiple LLMs with apps/APIs/data sources.
• Experience designing multi-agent architectures and managing agent fleets, context, memory.
• Hands-on with RAG systems and vector databases.
• Experience evaluating LLM response quality and agent behavior; building tests and datasets.
• Monitoring LLM and agent activity via logging and tracing to detect anomalies.
• Efficient performance and cost optimizations of LLM-based solutions.
• Security-focused testing: threat modelling and adversarial testing to protect safeguards.
• Experience deploying, operating, and scaling LLM-based solutions.
• Strong English and Polish communication.

Responsibilities

• Develop and scale AI-powered security features for passwordless authentication.
• Design multi-agent architectures and orchestrate agent collaboration.
• Build agent tools, manage context and memory.
• Implement RAG systems and embeddings workflows.
• Evaluate LLMs, create testing pipelines, ensure data quality.
• Monitor agent activity and detect suspicious behavior in apps.
• Optimize performance and costs through caching and model choice.
• Conduct threat modelling and adversarial testing to strengthen safeguards.
• Deploy and operate LLM-based solutions at scale.

Skills

Python

LLM integration

Multi-agent design

RAG systems

Embeddings & vector DBs

Performance optimization

Threat modelling

Adversarial testing

LLM monitoring

English & Polish

Tools

Garak

PromptFoo

Langfuse

Docker

Job description

About Us

At Framna, we partner with ambitious brands to create digital products that make a meaningful impact.

We work across the full product journey, from strategy and UX to engineering and long-term growth, combining technical expertise with a strong product mindset. Together with our partners, we build solutions that users rely on every day.

Our culture is built on trust, ownership, and fearless collaboration. We challenge ideas, support each other, and keep learning-openly sharing feedback, staying flexible in how we work, and approaching each other with empathy. Because the best products are created by teams that move forward together.

About the Role

You will work on AI-powered security solutions for a next-generation, passwordless authentication platform used by millions worldwide, leveraging advanced biometrics to protect digital identities.

Using Python and a range of LLMs, you will build and scale production solutions, including RAG and multi-agent systems. Your work will cover agent orchestration, tools and integrations, context and memory management, evaluation, and observability, with a focus on quality, reliability, performance, and cost efficiency.

Through controlled adversarial testing, you will investigate how attackers could use or manipulate LLMs and AI agents to bypass product safeguards, including in Android and iOS environments. You will analyse agent activity in customer applications, identify potentially unsafe or unauthorised behaviour, and translate findings into stronger safeguards and product improvements.

We value occasional in-person meetings at our studio to support collaboration and knowledge sharing.

Preferred Skills & Qualifications

• At least 7 years of experience in software engineering, machine learning engineering, or security engineering, including at least 2 years developing and maintaining LLM-based solutions in production

• Good knowledge of Python and experience building maintainable backend components that integrate multiple LLMs with applications, APIs, and data sources

• Experience designing multi-agent architectures, orchestrating agent collaboration, designing and building agent tools, and managing agent fleets, context, and memory.

• Hands-on experience building RAG systems and working with embeddings and vector databases

• Experience evaluating LLM response quality and agent behaviour, building evaluation datasets and automated testing pipelines, with an understanding of data quality, validation, and data leakage

• Experience monitoring LLM and agent activity in customer applications through logging, tracing, and traffic analysis to identify suspicious sources, malicious behaviour, and unauthorised activity.

• Experience optimising the performance and costs of LLM-based solutions through model selection, caching, and rate-limit management

• Practical experience investigating vulnerabilities in LLM-based applications and agentic systems through threat modelling and controlled adversarial testing, including identifying potential attack paths, attempting to bypass safeguards, and implementing protections against prompt injection, tool misuse, and abuse

• Experience deploying, operating, and scaling LLM-based solutions

• Very good communication skills in English and Polish
  

Nice to Have

• Experience in security research or red teaming beyond LLMs, particularly assessing Android or iOS applications and their backend integrations for vulnerabilities

• Familiarity with tools such as garak, promptfoo, or Langfuse

• Familiarity with Docker, CI/CD pipelines, and cloud services for deploying and scaling LLM-based applications

What you will find at Framna

• Real ownership and influence on the products you build

• A focus on business value over technical perfection

• Continuous feedback and open, honest communication

• Regular 1:1s and a mentoring program to support your growth

• Internal workshops and knowledge sharing across teams

• Flexible working hours (you can start between 7:00–10:00)

• Hybrid model with regular in-studio collaboration (Gdańsk, Wrzeszcz)

• High-quality equipment, including MacBook and testing devices

• Individual training budget

• English classes in small groups

Plus the essentials:

• Private medical care (Compensa) and MultiSport

• Team gatherings and shared moments beyond work

If you are motivated by ownership, collaboration, and continuous improvement, and you enjoy solving real problems with a team you can rely on—you will feel at home at Framna.

Let’s build impactful products together.