Forward Deployed Engineer, AI Evaluation & Production Quality Trida AI Boston, MA (Remote) Contract · 1 week ago

Trida Labs; Boston (MA); Remote; USD 124,000 - 186,000; Part time

Job summary

Trida AI deploys Forward Deployed Engineers inside enterprise customer teams to design, build, and ship production AI systems — agents, retrieval, and data integrations — and to leave behind systems the customer can operate themselves. Success is production adoption and measurable workflow impact, not a demo.

Travel to customer sites up to 30% may be required depending on the engagement. What You'll Do: Build eval harnesses: offline benchmarks, golden datasets and regression suites for customer

Qualifications

• Experience with production AI systems and agent workflows.
• Strong understanding of model behavior, safety, and latency/cost trade-offs.
• Ability to design robust evaluation criteria and defend thresholds.

Responsibilities

• Build eval harnesses: offline benchmarks, golden datasets and regression suites for customer AI systems.
• Define customer-specific acceptance criteria and launch-readiness thresholds.
• Implement tracing, observability, guardrails and monitoring for agents in production.
• Run model and framework upgrade evaluations — quality, safety, latency and cost.
• Make quality legible to customer stakeholders with evidence, not anecdotes.

Skills

Python programming

LLM observability

Model evaluation

Tools

Eval harnesses

Observability tooling

Tracing

Job description

Applications may be assessed in part using Trida's skill-based tools (verified certifications, skill tests, and FitScore), always alongside human review before any hiring decision.

Work Model:

Remote

Employment Type:

Contract

Experience:

5–10 Years

Compensation:

$90–$135 per hour

About Trida AI

Trida AI is a Forward Deployed Engineering services company that helps enterprises design, build, and deploy production AI systems. Our engineers embed with customer teams to ship agentic workflows, retrieval and data integrations, and the evaluation and hardening work that turns pilots into dependable production systems. Trida AI is expanding its services footprint across the United States as demand for production AI delivery grows.

About the Role

Trida AI deploys Forward Deployed Engineers inside enterprise customer teams to design, build, and ship production AI systems — agents, retrieval, and data integrations — and to leave behind systems the customer can operate themselves. Success is production adoption and measurable workflow impact, not a demo. Travel to customer sites up to 30% may be required depending on the engagement.

What You'll Do

Build eval harnesses: offline benchmarks, golden datasets and regression suites for customer AI systems

Define customer-specific acceptance criteria and launch-readiness thresholds

Implement tracing, observability, guardrails and monitoring for agents in production

Run model and framework upgrade evaluations — quality, safety, latency and cost

Make quality legible to customer stakeholders with evidence, not anecdotes

What We're Looking For

Production Python and hands-on eval-framework and LLM observability work

Real understanding of model behaviour, safety and latency/cost trade-offs

Statistical rigour: you design samples and thresholds you can defend

Additional Information

Must be authorized to work in the US; Trida AI does not provide sponsorship. Trida AI LLC is an equal opportunity employer. We welcome applicants of all backgrounds. If you need an accommodation in the application process, contact [e-mail]. Location: Remote — United States (Boston, MA base).

About Trida AI

Forward-deployed AI engineers embedded in your company to build, ship, and run AI systems that actually work in production.