AI Evaluation Engineer — Flexible Hours

HSBC Global Services Limited; Kraków; On-site; PLN 195,000 - 293,000; Full time

Benefits offered by this job

Healthcare

Pension

Life Insurance

Home office setup reimbursement (up to

Home office setup reimbursement (up to

Job summary

HSBC Global Services Limited is seeking an engineer to join a six-person team building flagship AI/ML projects for Credit and Lending. You will craft evaluation strategies, develop datasets, and ensure safety and reliability before production releases.

You will work with business stakeholders, engineers, and product managers to align evaluation coverage with user needs and strategic goals, in a fast-moving, international environment.

Qualifications

• Deep expertise in Python and ML/AI tooling.
• Experience with FastAPI, LangChain, and data containers.
• Solid understanding of microservices, observability, API design, and concurrency.
• Experience deploying on Azure or GCP with containers and Kubernetes.
• Hands-on with TDD/BDD and automated testing.
• Knowledge of RAG, prompt engineering, and agentic architectures.
• Clear written/spoken English for business partners.

Responsibilities

• Define and implement evaluation strategies for LLM/AI apps with clear criteria.
• Build/maintain evaluation datasets and test harnesses.
• Measure quality, relevance, latency, cost using automated and human methods.
• Evaluate data retrieval, grounding, attribution across financial data.
• Test AI systems for safety, bias, hallucinations; guide improvements.
• Automate evaluation and regression tests in CI/CD.

Skills

Python

ML/AI

FastAPI

LangChain

Data containers

Kubernetes

CI/CD

Azure DevOps

GCP

English

Tools

Python ecosystem

FastAPI

LangChain

Docker

Kubernetes

Jenkins

Azure DevOps

GCP Cloud Build

Job description

HSBC Global Services Limited is seeking an engineer to join a six-person team building flagship AI/ML projects for Credit and Lending. You will craft evaluation strategies, develop datasets, and ensure safety and reliability before production releases.

You will work with business stakeholders, engineers, and product managers to align evaluation coverage with user needs and strategic goals, in a fast-moving, international environment.