Remote AI Evaluation Engineer for Physical Systems

P-1 Ai; United States; Hybrid; USD 110,000 - 180,000; Full time

Benefits offered by this job

Competitive salary

Equity ownership

Healthcare

Unlimited PTO

Job summary

P-1 AI is pioneering an AI engineer agent for the physical world named Archie. You will help evaluate Archie against real engineering tasks, work with customers and partners, and shape evaluation methods that guide product development.

You will ensure the underlying engineering reasoning remains sound and actionable. The role sits at the forefront of AI for physical engineering, collaborating with cross-functional teams to expose failure modes and improve Archie’s capabilities toward engineering

Qualifications

• Technical background in a physical engineering domain.
• Strong understanding of agentic AI systems to design and reason about evaluations.
• Experience with AI evaluations, including creating evaluation tasks and judging methods.
• Comfortable working with customers, engineers, SMEs, partners, and internal teams to extract relevance from workflows.
• Strong product judgment, turning observations into concrete evaluation work.

Responsibilities

• Study how engineers and customers use Archie, identify successes and failures from an engineering perspective, and translate observations into actionable evaluations.
• Recreate real-world engineering tasks and failure modes for repeatable development use.
• Build and refine evaluation methodologies, including automated judges and human evaluation processes.
• Assess alignment between automated judges and expert human judgment and improve them when needed.
• Collaborate across engineering, product, customers, SMEs, and external partners to reflect real workflows in evaluations.

Skills

Physical engineering background

Agentic AI understanding

AI evaluation design

Customer collaboration

Product judgment

Job description

P-1 AI is pioneering an AI engineer agent for the physical world named Archie. You will help evaluate Archie against real engineering tasks, work with customers and partners, and shape evaluation methods that guide product development.

You will ensure the underlying engineering reasoning remains sound and actionable. The role sits at the forefront of AI for physical engineering, collaborating with cross-functional teams to expose failure modes and improve Archie’s capabilities toward engineering