Lead a pod of 5-10 trainers developing realistic, terminal-based scientific and technical tasks for Terminal Bench Science. Review trainer-created work for mathematical correctness, reproducibility, challenge, and grader reliability; mentor trainers; manage quality and throughput; and improve benchmark quality with the Mathematics Team Lead and technical stakeholders.
Responsibilities
- Review task statements, datasets, environments, reference solutions, and automated tests end-to-end.
- Verify correctness, well-posedness, convergence, stability, conditioning, discretization error, and edge cases. Ensure tasks require multi-step reasoning and use exact, symbolic, arbitrary-precision, or floating-point computation appropriately.
- Design and review graders based on residuals, feasibility, optimality certificates, invariants, error bounds, or proof checking.
- Ensure appropriate numerical tolerances, reproducibility through pinned tools and dependencies, and protection against hard-coded answers, test-file leakage, and bypasses.
- Give precise feedback, track revisions through closure, onboard and mentor trainers, allocate work, monitor quality and throughput, **escalate blockers**, and maintain pod documentation.
Requirements
- Ph. D., postdoctoral experience, or equivalent advanced experience in mathematics, applied mathematics, statistics, or a related field.
- Strong programming skills in Python, Julia, C/C++, MATLAB/Octave, or a computer algebra system; strong Linux and terminal proficiency.
- Hands-on experience with numerical methods, scientific computing, symbolic computation, optimization, computational statistics, or mathematical modeling.
- Ability to rigorously evaluate derivations, algorithms, proofs, computational results, research outputs, code, and scientific workflows.
- Experience mentoring or leading small technical or research teams; strong written communication and precise, constructive feedback.
- Ability to assess reproducibility, test reliability, and the scientific validity of computational tasks.
Nice to have
- Experience with NumPy, SciPy, SymPy, mpmath, SageMath, PARI/GP, GAP, Macaulay2, Mathematica-like workflows, FEniCS, Firedrake, DifferentialEquations.jl, PETSc, CVXPY, JuMP, Pyomo, HiGHS, Lean/mathlib, Coq/Rocq, Isabelle, Docker, Git, CI, automated testing, benchmarks, competition problems, computational evaluations, AI coding/reasoning/terminal agents, olympiad or competition problem setting/judging, publications or open-source work, or AI training/data-generation/model-evaluation programs.