Remote AI Engineer, Quality & Evaluation at Enterprise Scale

Fieldguide; San Francisco (CA); Hybrid; USD 120,000 - 160,000; Full time

Benefits offered by this job

Competitive compensation packages

Flexible PTO

401k

Wellness benefits including free therapy sessions

Technology & Work from Home reimbursement

Flexible work schedules

Job summary

A pioneering AI technology firm based in San Francisco is seeking an AI Engineer to own the evaluation infrastructure for AI agents. This role requires designing automated pipelines and building observability systems, ensuring agent performance meets enterprise standards. Ideal candidates have experience in production software and familiarity with TypeScript, React, and Python. The firm values collaboration, inclusivity, and personal growth, offering competitive compensation and flexible work arrangements.

Qualifications

• Multiple years of experience shipping production software in complex systems.
• Experience with TypeScript, React, Python, and Postgres.
• Built LLM-powered features serving production traffic.

Responsibilities

• Design and build a unified evaluation platform for agentic systems.
• Build observability systems to trace agent behavior and pinpoint failures.
• Own the evaluation infrastructure stack for seamless integration.

Skills

Production software experience

TypeScript

React

Python

Postgres

Evaluation frameworks

Observability infrastructure

Vector databases

Tools

LangSmith

Langfuse

Job description

A pioneering AI technology firm based in San Francisco is seeking an AI Engineer to own the evaluation infrastructure for AI agents. This role requires designing automated pipelines and building observability systems, ensuring agent performance meets enterprise standards. Ideal candidates have experience in production software and familiarity with TypeScript, React, and Python. The firm values collaboration, inclusivity, and personal growth, offering competitive compensation and flexible work arrangements.