Remote Physics Problem Designer & Evaluator

24-Mag Llc; New York (NY); Remote; USD 90,000 - 130,000; Full time

Job summary

24-MAG LLC is seeking advanced physics researchers to design and solve complex problems for evaluating and training large language models. The role involves creating rigorous, multi-step problems spanning undergraduate to PhD topics, and delivering clear explanations and annotated solutions.

Ideal candidates are pursuing or holding a Master's, Ph.D., or postdoctoral position in physics or related fields, with strong analytical skills and excellent written English.

Qualifications

• Master's/PhD/postdoc in Physics or related field.
• Strong foundation in advanced physics.
• Experience with quantum mechanics and quantum information is a plus.

Responsibilities

• Design challenging physics problems across advanced subject areas.
• Create tasks that probe limitations in scientific reasoning.
• Develop problems requiring abstraction and multi-step analysis.
• Incorporate symbolic manipulation and quantitative reasoning where appropriate.
• Maintain suitable difficulty across undergraduate through PhD-level topics.
• Solve complex physics problems accurately and systematically.
• Produce clear and rigorous step-by-step solutions.
• Explain assumptions, calculations, and intermediate reasoning explicitly.
• Apply structured physical reasoning to challenging scenarios.
• Present conclusions in a format suitable for evaluation and review.
• Evaluate model performance on physics-focused tasks.
• Identify recurring weaknesses in conceptual understanding.
• Analyse errors involving abstraction, symbolic manipulation, and multi-step reasoning.
• Distinguish conceptual mistakes from calculation or interpretation errors.
• Provide structured observations to support model improvement.
• Contribute to new physics evaluation benchmarks.
• Align problem design with research objectives.
• Develop tasks spanning different levels of technical complexity.
• Refine evaluation criteria based on observed model behaviour.
• Support consistent assessment across advanced physics domains.

Skills

Analytical and research capabilities

Remote collaboration skills

Explain complex physics clearly

Self-motivated and independent work

Strong English communication

Education

Master's/PhD/postdoc in Physics or related field

Strong foundation in advanced physics

Job description

24-MAG LLC is seeking advanced physics researchers to design and solve complex problems for evaluating and training large language models. The role involves creating rigorous, multi-step problems spanning undergraduate to PhD topics, and delivering clear explanations and annotated solutions.

Ideal candidates are pursuing or holding a Master's, Ph.D., or postdoctoral position in physics or related fields, with strong analytical skills and excellent written English.