Staff AI Engineer: Agent Performance & Product (Remote)

Remote; San Francisco (CA); Hybrid; USD 180,000 - 240,000; Full time

Benefits offered by this job

Remote-friendly culture

Professional development

Flexible hours

Job summary

Arcadia in San Francisco offers a staff-level AI role focused on building and refining agentic capabilities across real healthcare workflows. You will own product-layer decisions, including prompting, retrieval, memory, and escalation, while partnering with Product and Engineering to deliver steerable, trustworthy AI at scale.

The role emphasizes measurable improvements in accuracy, reliability, and cost, with an emphasis on safety, transparency, and patient impact.

Qualifications

• 8+ years of production software engineering experience with 3+ years in ML/LLM in production.
• Hands-on ownership of ML/LLM or agentic systems in production.
• Experience in healthcare, finance, or regulated industries.
• Familiarity with RAG architecture and evaluation frameworks.
• Experience with AWS AI/ML services (Bedrock, SageMaker).

Responsibilities

• Design and implement agent behavior across real workflows.
• Design retrieval/context architecture to feed models.
• Manage memory/state across multi-turn flows.
• Create prompts/templates for consistent agent behavior.
• Run evaluations against production conditions to measure performance.
• Define evaluation rubrics and thresholds for failures.
• Collaborate with Product and Engineering to ensure steerable and trustworthy agents.

Skills

Production software engineering

ML / LLM / agentic systems

Healthcare / regulated industries

AWS AI/ML services (Bedrock, SageMaker

Tools

AWS Bedrock

SageMaker

Job description

Arcadia in San Francisco offers a staff-level AI role focused on building and refining agentic capabilities across real healthcare workflows. You will own product-layer decisions, including prompting, retrieval, memory, and escalation, while partnering with Product and Engineering to deliver steerable, trustworthy AI at scale.

The role emphasizes measurable improvements in accuracy, reliability, and cost, with an emphasis on safety, transparency, and patient impact.