LLM & RAG Evaluation Engineer — Safe AI & Agent Testing

Axyde Analytics, LLC; Washington (District of Columbia); On-site; USD 130,000 - 190,000; Full time

Job summary

Axyde Analytics, LLC seeks a contingent LLM/RAG and Agent Evaluation Engineer to support ICE AI / Data Advisory responses. The role focuses on reproducible evaluation of AI systems for useful work, safety, reliability and cost in Washington, DC.

The opportunity is contingent on government award and funding. Responsibilities include evaluating retrieval quality, grounding, model routing, tool access and agent behavior; testing prompt injections and failure scenarios; capturing traceable evidence

Responsibilities

• Evaluate retrieval quality, grounding, model routing, tool access and agent behavior against defined baselines.
• Test prompt injection, prohibited access, model/provider failure, retries, budget exhaustion, escalation, human override and recovery in authorized environments.
• Capture versioned test inputs, configurations, results and traceable evidence; quantify quality, latency and cost tradeoffs.
• Explain findings and limitations to architecture, assurance and mission leads; recommend and verify targeted improvements.
• Hands‑on implementation and evaluation of LLM/RAG systems, with attributable work on agent/tool behavior or model routing.
• Ability to build repeatable tests and inspect logs, retrieval behavior, permissions and failure handling.
• Experience translating test results into operational decisions and recognizing limitations in test data and metrics.

Job description

Axyde Analytics, LLC seeks a contingent LLM/RAG and Agent Evaluation Engineer to support ICE AI / Data Advisory responses. The role focuses on reproducible evaluation of AI systems for useful work, safety, reliability and cost in Washington, DC.

The opportunity is contingent on government award and funding. Responsibilities include evaluating retrieval quality, grounding, model routing, tool access and agent behavior; testing prompt injections and failure scenarios; capturing traceable evidence