Senior AI Evaluation Engineer — Grounded & Trustworthy AI

Venture Up; Lisboa; Presencial; EUR 70 000 - 100 000; Tempo integral

Resumo da oferta

Venture Up in Lisbon is seeking a Senior Evaluation Engineer to ensure our platform is correct, private, grounded, and trustworthy before it reaches users.

You will define and execute evaluation across NLP, entity resolution, adversarial testing, grounded generation, and end-to-end data handling. Join a stealth AI startup shaping benchmarks and release readiness for real-time datasets.

Qualificações

• Experience evaluating NLP systems and extracting claims.
• Proven ability to resolve entities across datasets.
• Ability to design adversarial tests focused on privacy and security.
• Experience evaluating grounded generation and retrieval workflows.
• Design real-world benchmarks and gold standards.
• Experience reading formal technical specifications.

Responsabilidades

• Substrate testing — acceptance tests, invariants, privacy scenarios and test worlds.
• Actual knowledge — retrieval accuracy, citation faithfulness, evidence tracing.
• Information sharing — end-to-end testing of disclosure and private delivery.
• Derived knowledge — golden sets for detectors and thresholds.
• Evaluation discipline — held-out datasets, audit loops and threshold tuning.
• Launch readiness — define accuracy and latency bars for real-time data.

Conhecimentos

NLP evaluation

Entity resolution

Adversarial testing

Grounded generation evaluation

Benchmark design

Specification reading

Descrição da oferta de emprego

Venture Up in Lisbon is seeking a Senior Evaluation Engineer to ensure our platform is correct, private, grounded, and trustworthy before it reaches users.

You will define and execute evaluation across NLP, entity resolution, adversarial testing, grounded generation, and end-to-end data handling. Join a stealth AI startup shaping benchmarks and release readiness for real-time datasets.