Remote LLM Red Team Specialist - Failure Modes & Edge Cases

Weekday AI; United States; Remote; USD 171,924,000 - 257,887,000; Full time

Job summary

Weekday AI is seeking analytical and technically skilled professionals for a fully remote, full-time engagement focused on building evaluation benchmarks for frontier AI models. You will identify subtle failure modes, design multi-step tasks, and collaborate with researchers to improve robustness and safety.

Remote, independent contractor role requiring about 35 hours per week, with weekly payments. Candidates with background in AI safety, benchmarking, and reproducible experiments are

Qualifications

• Master’s degree, PhD, or equivalent practical experience in a STEM field with research, coding, or data analysis.
• Minimum 1 year of AI research, research engineering, security research, or related technical experience.
• Experience identifying vulnerabilities, adversarial behaviors, edge cases, or failure modes in LLMs or ML systems.
• Proficient in Python and Git; able to build custom scripts for experimentation, testing, and analysis.
• Solid understanding of modern LLMs, evaluation methodologies, and benchmarking.
• Excellent analytical thinking, creativity, and attention to detail; ability to work independently.
• Outstanding written communication skills.
• Ability to commit approximately 35 hours per week consistently.

Responsibilities

• Investigate how frontier AI models perform across coding, ML, analytical reasoning, and complex problem-solving tasks.
• Identify hidden failure modes, edge cases, reasoning errors, and vulnerabilities that may not be apparent through standard testing.
• Design challenging evaluation tasks that measure AI capabilities while remaining objective and reproducible.
• Document findings with clear technical explanations, supporting evidence, and reproducible methodologies.
• Collaborate with benchmark designers and AI researchers to refine evaluation tasks, eliminate loopholes, and strengthen grading criteria.
• Share insights and recommendations with cross-functional teams to improve AI evaluation quality and benchmark coverage.

Skills

Analytical thinking

Independent problem solving

Excellent written communication

Education

Master's degree or PhD in STEM

Tools

Python

Git

Job description

Weekday AI is seeking analytical and technically skilled professionals for a fully remote, full-time engagement focused on building evaluation benchmarks for frontier AI models. You will identify subtle failure modes, design multi-step tasks, and collaborate with researchers to improve robustness and safety.

Remote, independent contractor role requiring about 35 hours per week, with weekly payments. Candidates with background in AI safety, benchmarking, and reproducible experiments are