Senior Evaluation Engineer

Venture Up; Lisboa; Presencial; EUR 70 000 - 100 000; Tempo integral

Resumo da oferta

Venture Up in Lisbon is seeking a Senior Evaluation Engineer to ensure our platform is correct, private, grounded, and trustworthy before it reaches users.

You will define and execute evaluation across NLP, entity resolution, adversarial testing, grounded generation, and end-to-end data handling. Join a stealth AI startup shaping benchmarks and release readiness for real-time datasets.

Qualificações

• Experience evaluating NLP systems and extracting claims.
• Proven ability to resolve entities across datasets.
• Ability to design adversarial tests focused on privacy and security.
• Experience evaluating grounded generation and retrieval workflows.
• Design real-world benchmarks and gold standards.
• Experience reading formal technical specifications.

Responsabilidades

• Substrate testing — acceptance tests, invariants, privacy scenarios and test worlds.
• Actual knowledge — retrieval accuracy, citation faithfulness, evidence tracing.
• Information sharing — end-to-end testing of disclosure and private delivery.
• Derived knowledge — golden sets for detectors and thresholds.
• Evaluation discipline — held-out datasets, audit loops and threshold tuning.
• Launch readiness — define accuracy and latency bars for real-time data.

Conhecimentos

NLP evaluation

Entity resolution

Adversarial testing

Grounded generation evaluation

Benchmark design

Specification reading

Descrição da oferta de emprego

Senior Evaluation Engineer — Lisbon | Stealth AI Startup

We’re building a new kind of collective intelligence platform: AI that turns unstructured communication into grounded insights, recommendations, and actions.

We’re in stealth mode, but the challenge is anything but quiet.

We’re looking for a Senior Evaluation Engineer to make sure what we build is correct, private, grounded, and trustworthy — before it reaches the world.

You’ll be a great fit if you have experience with:

• NLP evaluation and claim extraction
• Entity resolution
• Adversarial / security-minded testing
• Grounded generation and retrieval evaluation
• Real-world benchmark and golden-set design
• Reading and working from formal technical specifications

You’ll own evaluation across:

• Substrate testing — acceptance tests, invariants, adversarial privacy/access scenarios and permission-asymmetric test worlds.
• Actual knowledge — retrieval accuracy, citation faithfulness, claim-to-evidence tracing, reasoning benchmarks and derivative-knowledge quality.
• Information sharing — end-to-end testing of disclosure, private delivery, audience boundaries and egress.
• Derived knowledge — golden sets for inference detectors, entity resolution, precision thresholds and feedback-driven evaluation.
• Evaluation discipline — held-out datasets, independent answer keys, human audit loops and governed threshold tuning.
• Launch readiness — defining the bar for accuracy, p95 latency and sustained throughput on large, real-time datasets.