Senior Legal AI Evaluation Architect (Remote)

24-Mag Llc; New York (NY); À distance; USD 400 000 - 800 000; Plein temps

Avantages offerts par ce poste

Health insurance

Paid time off

Retirement plan contributions

Equity incentives

Résumé du poste

24-MAG LLC is seeking experienced legal researchers to design evaluation frameworks for AI systems operating in legal domains. The role involves developing benchmark suites, methodologies, and datasets to assess legal reasoning and analytical quality.

You will collaborate with researchers and engineers on complex tasks spanning corporate law, contracts, and regulatory compliance, contributing to advanced AI evaluation initiatives.

Qualifications

• Advanced degree in Law required (JD/LLM/SJD/PhD or equivalent).
• Deep expertise in substantive legal domains and research methodology.
• Strong written and analytical abilities, capable of structured evaluation.

Responsabilités

• Design and own evaluation frameworks for legal AI systems.
• Develop benchmark suites, scoring methodologies and rubrics.
• Conduct applied research on legal reasoning and interpretation.
• Collaborate with researchers and engineers to assess AI performance.
• Create datasets and benchmark environments for evaluation.
• Analyze failures and provide structured improvement recommendations.

Connaissances

Legal research

Legal analysis

Policy work

Written communication

Interdisciplinary collaboration

Formation

JD/LLM/SJD/PhD in Law

Description du poste

24-MAG LLC is seeking experienced legal researchers to design evaluation frameworks for AI systems operating in legal domains. The role involves developing benchmark suites, methodologies, and datasets to assess legal reasoning and analytical quality.

You will collaborate with researchers and engineers on complex tasks spanning corporate law, contracts, and regulatory compliance, contributing to advanced AI evaluation initiatives.