Legal AI Annotator

Lexroom Srl; Milano; Remote; EUR 28,000 - 41,000; Part time

Job summary

Lexroom Srl is seeking qualified lawyers to join as independent contractors. You will evaluate AI-generated legal answers and define evaluation criteria, working remotely with a monthly in-person alignment meeting. This role leverages deep legal expertise across areas such as civil, corporate, or tax, and requires an active Italian VAT number and bar admission.

Italian language proficiency is essential. Compensation is €20 per hour, invoiced monthly based on hours worked.

Qualifications

• Must be a qualified lawyer admitted to the Italian Bar (Iscrizione all'Albo).
• Active Italian VAT number (Partita IVA).
• Excellent command of Italian (language requirement).
• Solid experience in at least one area of law (e.g., civil, criminal, administrative, banking & insurance, tax, corporate, employment, family, consumer, insolvency).

Responsibilities

• Legal AI benchmarks: evaluate AI-generated legal answers and define criteria for correctness.
• Arena (Comparative Evaluation): compare model outputs and select the best answer based on predefined criteria.

Education

Law degree (LL.B./J.D.)

Job description

At Lexroom, we set the bar for legal AI – clear, fast, and built for trust.

In just a few years, what began as an idea has become a reliable platform adopted by thousands of legal professionals across Europe.

Our mission is clear: make the law faster, clearer, and more reliable, giving lawyers, law firms, and companies the time and confidence to focus on what truly matters. We're scaling rapidly, backed by top-tier investors (having recently closed our Series B) and a world-class team dedicated to rigor, clarity, and impact.

Today, Lexroom is a trusted partner, shaping the future of legal AI with discipline and purpose.

Why Lexroom?

Shape the Standard: Be part of the team setting the benchmark for legal AI in Europe.

Work with the Best: Collaborate with exceptional colleagues in a culture that values competence, curiosity, and ambition.

Deliver Meaningful Results: Your legal judgment will directly shape the quality of answers thousands of professionals rely on every day.

At Lexroom, we leave buzzwords behind. Here, you'll build something that lasts: a trusted platform, a strong reputation, a new standard for the legal world.

The Role & Why Now

The quality of our legal answers is the core of Lexroom's services. To measure and continuously improve it, we're looking for lawyers who can put their expertise to work evaluating AI systems.

You’ll work independently and remotely on dedicated platforms, following Lexroom's guidelines, in direct contact with our People team.

Key Responsibilities

The scope of these activities may expand or change over time.

Legal AI Benchmarks: You’ll evaluate AI-generated answers to legal questions. You’ll also define the positive and negative criteria used to judge whether an answer is correct. Finally, you’ll build and review the evaluation sets we use to measure the quality of Lexroom's services.

Arena (Comparative Evaluation): You’ll compare alternative answers from different models to the same legal question. For each comparison, you’ll pick the preferred answer and explain your choice against predefined criteria: legal accuracy, completeness, use of sources, and clarity. Your evaluations directly guide product development.

Requirements

Bar Admission: You must be a qualified lawyer, already admitted to the Italian Bar (iscrizione all'Albo).

VAT Number: You need an active Italian VAT number (Partita IVA), as the collaboration is invoiced.

Legal Expertise: You should have solid experience in at least one area of law, to be specified in your CV. Examples include civil, criminal, administrative, banking & insurance, tax, corporate, employment, family, consumer, and insolvency law.

Language: Excellent command of Italian is required.

Nice to Have

These are a plus, but not required:

• familiarity with legal tech or generative AI tools in legal work;
• experience evaluating AI systems, such as benchmarks, A/B tests, or model comparisons.

Contract & Compensation

Contract: Service agreement as an independent contractor with a VAT number.

Compensation: €20/hour, invoiced monthly based on the hours actually worked and reported.

Work Setup: Fully remote, with one in-person meeting per month for alignment and goal-setting.

Hiring Process

The process is a single 30-minute introductory interview.

At Lexroom, ambition meets discipline and purpose. We're not just building a product—we're setting a new standard for the legal world. If you're ready to contribute to something lasting and meaningful, we'd love to hear from you.