Technical Lead, AI Evaluation & LLM Platform

InPost UK; Kraków; Hybrid; PLN 320,000 - 420,000; Full time

Benefits offered by this job

Hybrid work from Warsaw or Kraków

Flexible hours

B2B contract

Training opportunities

Access to frontier models

Open-source track

Job summary

InPost Group, the team behind Von Halsky, seeks a Technical Lead, Senior AI Engineer to own the Evaluations Platform and guide the next phase of production-grade AI tooling. You will oversee the LLM-judge, curate eval datasets, and drive measurable improvements in conversation quality.

You will mentor engineers, define the technical direction, and collaborate with PM/EM to deliver impactful outcomes in a hybrid setup from Poland’s key cities.

Qualifications

• 5+ years building and running production software with 2+ years on LLM-based systems.
• Strong engineering fundamentals and production ownership habits (testing, CI/CD, containers, observability, cloud).
• Experience evaluating generative systems: LLM-as-judge, calibration, and regression suites.
• AI engineering fundamentals: prompt/context engineering, tool use, multi-turn state, and observability tooling.
• Data at scale: SQL and data lakes/warehouses with reproducible metrics.
• Fluent English (B2) and Polish for user-conversations and disclosures.

Responsibilities

• Own the LLM-judge pipeline and eval-driven development.
• Develop eval datasets and maintain a comprehensive golden set.
• Perform root-cause analysis on real conversations and improve diagnosis.
• Maintain the interface to product and business with measurable, actionable results.
• Mentor engineers and help define technical direction for the platform.

Skills

Production software

LLM systems

CI/CD

Python

Observability

Cloud

SQL

English B2

Polish

AI eval tooling

Tools

LangFuse

Braintrust

Weave

Claude Code

Cursor

Copilot

Job description

InPost Group, the team behind Von Halsky, seeks a Technical Lead, Senior AI Engineer to own the Evaluations Platform and guide the next phase of production-grade AI tooling. You will oversee the LLM-judge, curate eval datasets, and drive measurable improvements in conversation quality.

You will mentor engineers, define the technical direction, and collaborate with PM/EM to deliver impactful outcomes in a hybrid setup from Poland’s key cities.