AI Evaluation Engineer

Crossing Hurdles; United States; Remote; USD 120,000 - 180,000; Full time

Job summary

Crossing Hurdles seeks a senior engineer to design and validate complex simulation problems for AI agents. You will create tasks with multiple constraints, run open-source tools in Python, and test solutions for accuracy and physical plausibility.

The role requires advanced engineering degrees and 10+ years hands-on design experience, plus strong documentation and analytical skills. You will analyze model outputs and iterate benchmarks to improve evaluation results.

Qualifications

• Masters or PhD in EE/ME/AE with 10+ years hands-on design experience.
• Proficient with engineering simulation tools and Python scripting.
• Experience with LLMs, coding agents, AI evaluation, or failure-mode analysis.
• Strong grasp of engineering principles, unit consistency, and validation.

Responsibilities

• Design and validate complex engineering simulations for AI agents.
• Create constrained optimization tasks with validated solutions.
• Build and test problems using open-source tools and Python.
• Analyze agent outputs and logs to identify failures and improve benchmarks.
• Ensure accuracy and physical plausibility of benchmark tasks.

Skills

Engineering simulation

Python scripting

AI evaluation

Failure-mode analysis

Education

Master's degree in EE/ME/AE

PhD in related field

Tools

Open-source simulation tools

Python tooling

Job description

Role & responsibilities

• Design and validate complex engineering simulation problems for AI agents.
• Create engineering design tasks with multiple constraints, optimization goals, and validated solutions.
• Use open-source simulation tools and Python to build and test engineering problems.
• Analyze AI agent outputs and execution logs to identify reasoning, coding, and simulation failures.
• Refine benchmark tasks based on model performance and ensure technical accuracy and physical plausibility.

Preferred candidate profile

• Masters degree or PhD in Electrical, Mechanical, or Aerospace Engineering with 10+ years of hands‑on engineering design experience.
• Strong experience with engineering simulation tools and Python scripting.
• Experience with LLMs, coding agents, AI evaluation, or failure‑mode analysis.
• Strong understanding of engineering principles, physical constraints, unit consistency, and simulation validation.
• Excellent analytical, problem‑solving, and technical documentation skills.