Remote AI Evaluation Engineer for Physical Engineering

Fdetimes; United States; Hybrid; USD 140,000 - 190,000; Full time

Benefits offered by this job

Healthcare

Dental

Vision

401(k) match

Equity ownership

Unlimited PTO

Job summary

P-1 AI seeks an AI evaluation engineer to sit at the frontier of AI for physical engineering. You will work with engineers, customers, partners, and SMEs to challenge Archie on tough tasks, understand where it fails, and capture those failures in actionable evaluations guiding product development.

You will design and run evaluations, recreate real-world engineering challenges, and refine methods to measure alignment with expert human judgment. Remote-friendly with on-site options.

Qualifications

• Technical background in a physical engineering domain (e.g., power systems, robotics, mechanical/electrical).
• Experience with AI evaluations, including creating tasks and comparing automated vs human judgment.
• Comfortable working with customers, SMEs, partners, and internal teams.

Responsibilities

• Study how engineers and customers use Archie; identify success/failure from an engineering view.
• Recreate real-world tasks and failure modes for reusable evaluation environments.
• Build and refine evaluation methodologies, incl. automated judges and human evaluation processes.
• Assess alignment between automated judges and expert human judgment.

Skills

Physical engineering background

Agentic AI understanding

AI evaluation experience

Customer-facing

Product judgment

Superintelligence mission

Education

Bachelor's degree in engineering or related field

Tools

Evaluation design

Job description

P-1 AI seeks an AI evaluation engineer to sit at the frontier of AI for physical engineering. You will work with engineers, customers, partners, and SMEs to challenge Archie on tough tasks, understand where it fails, and capture those failures in actionable evaluations guiding product development.

You will design and run evaluations, recreate real-world engineering challenges, and refine methods to measure alignment with expert human judgment. Remote-friendly with on-site options.