Remote Forward-Deployed AI Engineer: Evaluation & Production Quality

Trida AI; Boston (MA); Remote; USD 124,000 - 186,000; Part time

Job summary

Trida AI deploys Forward Deployed Engineers inside enterprise customer teams to design, build, and ship production AI systems — agents, retrieval, and data integrations — and to leave behind systems the customer can operate themselves. Success is production adoption and measurable workflow impact, not a demo.

Travel to customer sites up to 30% may be required depending on the engagement. What You'll Do: Build eval harnesses: offline benchmarks, golden datasets and regression suites for customer

Qualifications

• Experience with production AI systems and agent workflows.
• Strong understanding of model behavior, safety, and latency/cost trade-offs.
• Ability to design robust evaluation criteria and defend thresholds.

Responsibilities

• Build eval harnesses: offline benchmarks, golden datasets and regression suites for customer AI systems.
• Define customer-specific acceptance criteria and launch-readiness thresholds.
• Implement tracing, observability, guardrails and monitoring for agents in production.
• Run model and framework upgrade evaluations — quality, safety, latency and cost.
• Make quality legible to customer stakeholders with evidence, not anecdotes.

Skills

Python programming

LLM observability

Model evaluation

Tools

Eval harnesses

Observability tooling

Tracing

Job description

Trida AI deploys Forward Deployed Engineers inside enterprise customer teams to design, build, and ship production AI systems — agents, retrieval, and data integrations — and to leave behind systems the customer can operate themselves. Success is production adoption and measurable workflow impact, not a demo.

Travel to customer sites up to 30% may be required depending on the engagement. What You'll Do: Build eval harnesses: offline benchmarks, golden datasets and regression suites for customer