Expert LLM Engineer (AI Security)

Hub; Województwo pomorskie; Hybrid; PLN 240,000 - 360,000; Full time

Benefits offered by this job

Hybrid work model

Private medical care

Training budget

English classes

Mentoring program

Job summary

Framna is developing AI-powered security solutions for a next‑generation, passwordless authentication platform used by millions. You will work with Python and LLMs to prepare data, fine-tune models, and integrate with APIs and product tools.

You will test for quality, reliability, and cost efficiency while exploring adversarial risks across mobile environments and translating findings into product improvements.

Qualifications

• At least 7 years of experience in software engineering, machine learning engineering, or security engineering.
• 2+ years of hands-on experience developing and maintaining LLM-based solutions in production.
• Good knowledge of Python and ability to build maintainable, testable software.
• Experience building backend components and integrating models with applications, APIs, and data sources.
• Understanding of model evaluation and validation, including data quality and data leakage.
• Experience with prompt engineering and managing the context provided to models.
• Experience building AI agents, including tool calling and orchestrating multi-step workflows.
• Experience designing evaluation datasets and regression tests for LLM-based solutions.
• Practical understanding of security risks related to LLMs and AI agents, with testing for manipulation and bypass safeguards.
• Ability to analyse traffic, telemetry, or behavioural patterns to investigate automated activity.

Responsibilities

• Develop and evaluate AI-powered security solutions for a passwordless authentication platform.
• Prepare data, fine-tune models, and manage prompts and context with APIs and product tools.
• Build evaluation datasets and regression tests to improve quality and cost efficiency.
• Conduct controlled adversarial testing to identify weaknesses in safeguards.
• Investigate threats across Android and iOS environments and translate findings into product improvements.
• Collaborate in occasional in-person studio meetings to share knowledge.

Skills

Python

LLM

Security engineering

Machine learning engineering

Backend development

Data analysis

Prompt engineering

AI agents

Testing

Data quality

Tools

garak

promptfoo

Langfuse

Docker

CI/CD

LoRA/QLoRA

PEFT

Job description

About Us

At Framna, we partner with ambitious brands to create digital products that make a meaningful impact.

We work across the full product journey, from strategy and UX to engineering and long-term growth, combining technical expertise with a strong product mindset. Together with our partners, we build solutions that users rely on every day.

Our culture is built on trust, ownership, and fearless collaboration. We challenge ideas, support each other, and keep learning-openly sharing feedback, staying flexible in how we work, and approaching each other with empathy. Because the best products are created by teams that move forward together.

About the Role

You will work on AI-powered security solutions for a next-generation, passwordless authentication platform used by millions worldwide, leveraging advanced biometrics to protect digital identities.

Using Python and LLMs, you will develop and evaluate solutions—from data preparation and fine-tuning to prompting, context management, and integration with APIs and product tools. You will build evaluation datasets and regression tests to improve quality, reliability, performance, and cost efficiency.

Through controlled adversarial testing, you will explore how LLMs and AI agents can be manipulated or used to bypass product safeguards. You will investigate threats across Android and iOS environments, analyse agent behaviour in customer applications, and translate findings into product improvements.

We value occasional in-person meetings at our studio to support collaboration and knowledge sharing.

Preferred Skills & Qualifications

• At least 7 years of experience in software engineering, machine learning engineering, or security engineering, including at least 2 years of hands‑on experience developing and maintaining LLM‑based solutions in production

• Good knowledge of Python and the ability to build maintainable, testable software

• Experience building backend components and integrating models with applications, APIs, and data sources

• Understanding of model evaluation and validation principles, including data quality and data leakage between datasets

• Experience with prompt engineering and managing the context provided to models

• Experience building AI agents, including tool calling and orchestrating multi‑step workflows

• Experience designing evaluation datasets and regression tests to assess the quality, reliability, and safety of LLM‑based solutions

• Practical understanding of security risks related to LLMs and AI agents, with experience testing their resistance to manipulation and attempts to bypass safeguards

• Ability to analyse traffic, telemetry, or behavioural patterns to investigate automated activity and identify potentially unsafe or unauthorised actions

Nice to Have

• Experience fine‑tuning language models, including data preparation, selecting tuning approaches, and evaluating results

• Experience developing fraud or abuse detection mechanisms based on traffic, telemetry, or behavioural patterns

• Experience investigating mobile application security in Android or iOS environments

• Experience in security research, red teaming, or penetration testing beyond LLM‑based systems

• Familiarity with tools for LLM security testing, evaluation, and observability, such as garak, promptfoo, or Langfuse

• Familiarity with RAG, embeddings, and vector databases

• Knowledge of LoRA/QLoRA and libraries such as PEFT or TRL

• Familiarity with Docker, CI/CD, and cloud services used to deploy AI solutions

What you will find at Framna

• Real ownership and influence on the products you build

• A focus on business value over technical perfection

• Continuous feedback and open, honest communication

• Regular 1:1s and a mentoring program to support your growth

• Internal workshops and knowledge sharing across teams

• Flexible working hours (you can start between 7:00–10:00)

• Hybrid model with regular in-studio collaboration (Gdańsk, Wrzeszcz)

• High‑quality equipment, including MacBook and testing devices

• Individual training budget

• English classes in small groups

Plus the essentials:

• Private medical care (Compensa) and MultiSport

• Team gatherings and shared moments beyond work

If you are motivated by ownership, collaboration, and continuous improvement, and you enjoy solving real problems with a team you can rely on—you will feel at home at Framna. Let’s build impactful products together.