Senior AI Architect: LLMs, Multi-Agent Systems & RAG

Robots and Pencils; United States; Remote; USD 146,000 - 202,000; Full time

Job summary

Robots & Pencils is seeking a senior AI architecture lead to design end-to-end LLM and agentic systems for diverse client engagements. You’ll own architecture decisions across retrieval, RAG, and model serving on AWS, balancing accuracy, latency, and cost.

You’ll mentor engineers, define LLMOps standards, implement guardrails and observability, and collaborate with product and data teams to translate business goals into scalable AI solutions.

Qualifications

• 7+ years in software engineering with AI/ML leadership
• Deep expertise in LLM and agentic architecture
• Production RAG and retrieval systems design
• Model serving and inference optimization on AWS
• LLMOps and MLOps including eval pipelines & CI/CD
• Experience with AI evaluation, guardrails, and tracing tooling
• Proficiency in Python and at least one other language
• Strong cloud-native architecture knowledge (microservices, serverless)
• Understanding of responsible AI governance and data isolation

Responsibilities

• Define AI system architecture and lead design across LLM applications and agentic systems
• Drive RAG and context-engineering strategy (hybrid retrieval, embedding models)
• Lead multi-agent orchestration design using LangGraph, Bedrock Agents, or AgentCore
• Own model serving and inference architecture on AWS, optimizing latency and cost
• Define LLMOps and MLOps standards across engagements (versioning, CI/CD)
• Drive evaluation, guardrails, and observability strategies with offline/online evals
• Lead governance for responsible AI and data isolation measures
• Guide build choices (fine-tuning vs prompting vs RAG) with cost/maintainability in mind
• Collaborate with leadership and clients on AI technical direction
• Communicate complex AI concepts clearly to engineers and non-engineers
• Mentor engineers and align AI architecture with broader business goals
• Develop and maintain AI architecture docs and reusable patterns

Skills

LLM architecture

Multi-agent orchestration

RAG and retrieval design

AWS model serving

LLMOps & MLOps

Observability & eval pipelines

Python & other languages

Cloud-native architecture

Responsible AI governance

Leadership & mentoring

Tools

Claude

Cursor

Job description

Robots & Pencils is seeking a senior AI architecture lead to design end-to-end LLM and agentic systems for diverse client engagements. You’ll own architecture decisions across retrieval, RAG, and model serving on AWS, balancing accuracy, latency, and cost.

You’ll mentor engineers, define LLMOps standards, implement guardrails and observability, and collaborate with product and data teams to translate business goals into scalable AI solutions.