Senior AI Engineer, Evaluation & Reliability (Remote)

Fieldguide; San Francisco (CA); On-site; USD 200,000 - 250,000; Full time

Benefits offered by this job

Competitive compensation packages

Flexible PTO

401k

Wellness benefits

Technology & Work from Home reimbursement

Flexible work schedules

Job summary

Fieldguide, a remote-first company based in San Francisco, is seeking a Senior AI Engineer to lead the evaluation infrastructure for AI agents. This role focuses on enhancing the reliability and performance of AI systems to support top accounting and consulting firms.

The ideal candidate will have experience with TypeScript, Python, and observability systems, working collaboratively to design and build robust evaluation platforms. Fieldguide offers competitive compensation ranging from $200K to $250K, flexible PTO, and wellness benefits.

Qualifications

• Multiple years of experience shipping production software in complex, real-world systems.
• Experience with AI/ML systems and evaluation frameworks.
• Comfort operating in ambiguity and taking responsibility for outcomes.

Responsibilities

• Design and build a unified evaluation platform for agentic systems.
• Implement comparison frameworks to measure effectiveness and quality.
• Ensure reliability of AI agents at enterprise scale.

Skills

TypeScript

React

Python

Postgres

LLM evaluation frameworks

Observability systems

Data-driven decision making

Tools

LangSmith

Langfuse

Job description

Fieldguide, a remote-first company based in San Francisco, is seeking a Senior AI Engineer to lead the evaluation infrastructure for AI agents. This role focuses on enhancing the reliability and performance of AI systems to support top accounting and consulting firms.

The ideal candidate will have experience with TypeScript, Python, and observability systems, working collaboratively to design and build robust evaluation platforms. Fieldguide offers competitive compensation ranging from $200K to $250K, flexible PTO, and wellness benefits.