Legal AI Annotator

US Health Partners, LLC; Lombardia; Hybrid; EUR 23,000 - 32,000; Full time

Job summary

Lexroom is seeking qualified lawyers in Italy to evaluate AI-generated legal answers, setting benchmarks for accuracy, completeness, and reliability.

You will work independently and remotely on dedicated platforms, following our guidelines and staying in direct contact with our People team.

Contract is as an independent contractor with a VAT number, compensation is €20/hour invoiced monthly, with one in-person meeting per month for alignment.

Qualifications

• Bar admission to the Italian Bar is required.
• Active Partita IVA (VAT number) is needed for invoicing.
• Strong legal expertise in at least one area of law.

Responsibilities

• Legal AI benchmarks: evaluate AI-generated legal answers and define criteria for correctness.
• Arena/comparative evaluation: compare model outputs and justify your preferred answer.
• Contribute to product development by providing structured feedback on AI outputs.

Skills

Italian language proficiency

Legal analysis

Attention to detail

Job description

At Lexroom, we set the bar for legal AI – clear, fast, and built for trust.

In just a few years, what began as an idea has become a reliable platform adopted by thousands of legal professionals across Europe.

Our mission is clear: make the law faster, clearer, and more reliable, giving lawyers, law firms, and companies the time and confidence to focus on what truly matters. We're scaling rapidly, backed by top-tier investors (having recently closed our Series B) and a world-class team dedicated to rigor, clarity, and impact.

Today, Lexroom is a trusted partner, shaping the future of legal AI with discipline and purpose.

Why Lexroom?

Shape the Standard: Be part of the team setting the benchmark for legal AI in Europe.

Work with the Best: Collaborate with exceptional colleagues in a culture that values competence, curiosity, and ambition.

Deliver Meaningful Results: Your legal judgment will directly shape the quality of answers thousands of professionals rely on every day.

At Lexroom, we leave buzzwords behind. Here, you'll build something that lasts: a trusted platform, a strong reputation, a new standard for the legal world.

The Role & Why Now

The quality of our legal answers is the core of Lexroom's services. To measure and continuously improve it, we're looking for lawyers who can put their expertise to work evaluating AI systems.

You’ll work independently and remotely on dedicated platforms, following Lexroom's guidelines, in direct contact with our People team.

Key Responsibilities

The scope of these activities may expand or change over time.

Legal AI Benchmarks: You’ll evaluate AI-generated answers to legal questions. You’ll also define the positive and negative criteria used to judge whether an answer is correct. Finally, you’ll build and review the evaluation sets we use to measure the quality of Lexroom's services.

Arena (Comparative Evaluation): You’ll compare alternative answers from different models to the same legal question. For each comparison, you’ll pick the preferred answer and explain your choice against predefined criteria: legal accuracy, completeness, use of sources, and clarity. Your evaluations directly guide product development.

Requirements

Bar Admission: You must be a qualified lawyer, already admitted to the Italian Bar (iscrizione all'Albo).

VAT Number: You need an active Italian VAT number (Partita IVA), as the collaboration is invoiced.

Legal Expertise: You should have solid experience in at least one area of law, to be specified in your CV. Examples include civil, criminal, administrative, banking & insurance, tax, corporate, employment, family, consumer, and insolvency law.

Language: Excellent command of Italian is required.

Nice to Have

These are a plus, but not required:

• familiarity with legal tech or generative AI tools in legal work;
• experience evaluating AI systems, such as benchmarks, A/B tests, or model comparisons.

Contract & Compensation

Contract: Service agreement as an independent contractor with a VAT number.

Compensation: €20/hour, invoiced monthly based on the hours actually worked and reported.

Work Setup: Fully remote, with one in-person meeting per month for alignment and goal-setting.

Hiring Process

The process is a single 30-minute introductory interview.

At Lexroom, ambition meets discipline and purpose. We're not just building a product—we're setting a new standard for the legal world. If you're ready to contribute to something lasting and meaningful, we'd love to hear from you.