26-0098-003 | LLM / RAG and Agent Evaluation Engineer — ICE AI / Data Advisory (Contingent)

Axyde Analytics, LLC; Washington (District of Columbia); On-site; USD 130,000 - 190,000; Full time

Job summary

Axyde Analytics, LLC seeks a contingent LLM/RAG and Agent Evaluation Engineer to support ICE AI / Data Advisory responses. The role focuses on reproducible evaluation of AI systems for useful work, safety, reliability and cost in Washington, DC.

The opportunity is contingent on government award and funding. Responsibilities include evaluating retrieval quality, grounding, model routing, tool access and agent behavior; testing prompt injections and failure scenarios; capturing traceable evidence

Responsibilities

• Evaluate retrieval quality, grounding, model routing, tool access and agent behavior against defined baselines.
• Test prompt injection, prohibited access, model/provider failure, retries, budget exhaustion, escalation, human override and recovery in authorized environments.
• Capture versioned test inputs, configurations, results and traceable evidence; quantify quality, latency and cost tradeoffs.
• Explain findings and limitations to architecture, assurance and mission leads; recommend and verify targeted improvements.
• Hands‑on implementation and evaluation of LLM/RAG systems, with attributable work on agent/tool behavior or model routing.
• Ability to build repeatable tests and inspect logs, retrieval behavior, permissions and failure handling.
• Experience translating test results into operational decisions and recognizing limitations in test data and metrics.

Job description

26-0098-003 | LLM / RAG and Agent Evaluation Engineer — ICE AI / Data Advisory (Contingent)

• Washington, DC

Contingent on government contract award and funding.

Washington, DC is the opportunity-market location; actual work location and work model are to be determined.

Axyde Analytics, LLC is identifying experienced professionals for a proposed management and technical advisory team supporting its response to the ICE Enterprise AI, Data, and Technology Strategy/Architecture RFI (RFI-ICE_ADTS_Sep2026). The proposed mission is to help ICE turn AI and data investments into measurable operational results: establish a baseline, assess alternatives and cost/risk, recommend decisions, build an executable plan and verify benefits. This is a pre-award talent search for potential work; an RFI is market research and does not establish an awarded contract or guaranteed position.

Your contribution

Produce reproducible technical evidence about whether AI systems perform useful work safely, reliably and at an acceptable cost. Translate mission questions into controlled evaluation criteria and explain what the evidence supports.

Responsibilities

• Evaluate retrieval quality, grounding, model routing, tool access and agent behavior against defined baselines.
• Test prompt injection, prohibited access, model/provider failure, retries, budget exhaustion, escalation, human override and recovery in authorized environments.
• Capture versioned test inputs, configurations, results and traceable evidence; quantify quality, latency and cost tradeoffs.
• Explain findings and limitations to architecture, assurance and mission leads; recommend and verify targeted improvements.
• Hands‑on implementation and evaluation of LLM/RAG systems, with attributable work on agent/tool behavior or model routing.
• Ability to build repeatable tests and inspect logs, retrieval behavior, permissions and failure handling.
• Experience translating test results into operational decisions and recognizing limitations in test data and metrics.

Evidence we will discuss

• An existing candidate‑owned, public or authorized demonstration or redacted test report with rerunnable methodology.
• Evidence of safe failure, denied actions, recovery and cost/retry controls, with your specific contribution identified.

Technical context

The RFI describes an existing STELLA environment spanning AWS, Microsoft Azure and Google Cloud, with Kubernetes and LLM/RAG capabilities, and a data environment integrating Databricks, Collibra, MuleSoft and Traceable. Its enterprise control plane, agentic orchestration/runtime and independent assurance layers are future‑state concepts to be evaluated. Direct STELLA experience is preferred when lawfully demonstrable; comparable federal or regulated production experience is welcome. No applicant is expected to master every listed product or all three clouds.

Engagement conditions

Contingent on government contract award and funding. Work location, work model, travel and access requirements will be determined by the scope and agency requirements. Start date, engagement structure, compensation and any paid work terms will be agreed before work begins. This posting represents a coverage function, not a promise of a separate full‑time position. Candidates may cover multiple functions where experience, availability and independent-review requirements permit.