Remote LLM Red Team Specialist — Failure Modes

mercor; United States; Remoto; USD 83.000 - 124.000; Tempo pieno

Descrizione del lavoro

Mercor is seeking an LLM Red Team Specialist — Failure Modes & Edge Cases for remote work. The contract role requires about 35 hours per week, focusing on probing frontier models for vulnerabilities and crafting challenging, fair tasks with clear reproducible findings.

You will document results, close loopholes with researchers, and contribute to continuousbenchmark improvements in AI evaluation. Proficiency in Python, Git, and understanding LLM capabilities is essential.

Competenze

• MSc or PhD in a STEM field or equivalent practical experience.
• 1+ years in research, research-engineering, security, or AI-evaluation roles.
• Ability to identify vulnerabilities, edge cases, or failure modes in LLMs or ML systems.
• Proficiency in Python and Git for scripting probes and analyses.
• Familiarity with LLM capabilities, limitations, and evaluation techniques.
• Ability to engage reliably for ~35 hours/week.

Mansioni

• Probe models to explore how frontier AI models handle coding, ML, and analysis tasks.
• Design challenges by turning identified weaknesses into well-crafted tasks that are challenging for models but fair to grade.
• Document findings clearly with reproducible evidence and steps.
• Strengthen tasks by collaborating with task authors to close loopholes, shortcuts, and grading gaps.
• Work as a team by sharing insights with researchers and fellow experts to continually improve the benchmark.

Conoscenze

Python

Git

LLM evaluation

Vulnerability detection

Edge cases

Written communication

Formazione

MSc or PhD in STEM

Equivalent practical experience

Strumenti

Git

Descrizione del lavoro

Mercor is seeking an LLM Red Team Specialist — Failure Modes & Edge Cases for remote work. The contract role requires about 35 hours per week, focusing on probing frontier models for vulnerabilities and crafting challenging, fair tasks with clear reproducible findings.

You will document results, close loopholes with researchers, and contribute to continuousbenchmark improvements in AI evaluation. Proficiency in Python, Git, and understanding LLM capabilities is essential.