AI Evaluation Engineer: LLM Safety & Quality

HSBC Holdings plc; Województwo podlaskie; On-site; PLN 195,000 - 293,000; Full time

Benefits offered by this job

Multisport card

Private medical care

Life insurance

One-time reimbursement of home office"

Cafeteria platform

Employee assistance program

Additional contributions to PPK scheme

Corporate parties & events

CSR initiatives

Nursery discounts

Financial support with trainings and教育

Social fund

Flexible working hours

Free parking

Job summary

HSBC Holdings plc is seeking engineers to join a focused team working on flagship AI/ML projects for Credit and Lending. You will build robust evaluation approaches for LLM and AI applications to ensure quality, reliability and safety before production.

You will design evaluation datasets, automate tests, and collaborate with stakeholders to drive trustworthy AI solutions in a fast-moving environment. This role emphasizes hands-on Python/ML work and cloud deployments.

Qualifications

• Deep expertise in Python and its ML/AI ecosystem.
• Experience with FastAPI, LangChain and data containerisation.
• Understanding of microservices, observability, API design and concurrency models.
• Experience building deployments on Azure or GCP with containers and CI/CD tools.

Responsibilities

• Define evaluation strategies for LLM/AI in Credit and Lending with clear quality criteria.
• Build and maintain evaluation datasets and test harnesses for prompts, models, Retrieval-Augmented Generation and agentic workflows.
• Measure quality metrics: correctness, relevance, groundedness, latency and cost using automated and human methods.
• Evaluate retrieval quality, source attribution and grounding across complex financial data and vector databases.
• Test AI systems for robustness, hallucinations, bias, fairness and prompt injections; guide improvements.
• Automate evaluation and regression tests in CI/CD to assess changes before release.
• Analyse results, identify regressions and communicate trade-offs to engineering and business stakeholders.
• Collaborate with engineers, product managers and business experts to align evaluation with user needs.

Skills

Python ML ecosystem

FastAPI

LangChain

Data containerisation

Microservices observability

API design

Concurrency models

Azure/GCP deployment

Tools

Jenkins

Azure DevOps

GCP Cloud Build

Kubernetes

Job description

HSBC Holdings plc is seeking engineers to join a focused team working on flagship AI/ML projects for Credit and Lending. You will build robust evaluation approaches for LLM and AI applications to ensure quality, reliability and safety before production.

You will design evaluation datasets, automate tests, and collaborate with stakeholders to drive trustworthy AI solutions in a fast-moving environment. This role emphasizes hands-on Python/ML work and cloud deployments.