Technical Lead, AI Platform Architect

Ideaon Inc; San Francisco (CA); On-site; USD 240,000 - 320,000; Full time

Job summary

Ideaon Inc in San Francisco, CA, is seeking a Proven Technical Leader to architect and operate production backend and distributed systems. The role emphasizes hands-on LLM deployment, Python-based framework design, and deep expertise in distributed architectures.

You'll lead multi‑agent orchestration, define open protocols for interoperability, and drive observability, security, and scalable infrastructure across enterprise AI platforms. Requires 8+ years in backend leadership and a CS degree.

Qualifications

• 8+ years designing and operating production backend or distributed systems.
• 2+ years in a Technical Lead role.
• Bachelor’s degree in CS or equivalent experience.
• Hands-on experience deploying LLM-powered apps to production.
• Proficient in Python; building developer frameworks or APIs.

Responsibilities

• Architect tool and agent orchestration for multi-agent workflows.
• Design open protocols for interoperability of agents and tools.
• Develop AI user experiences with streaming outputs.
• Implement Retrieval-Augmented Generation and memory systems.
• Ensure observability with tracing, metrics, and SLOs.

Skills

Python

Distributed Systems

LLM Deployment

Agent Orchestration

API Design

Education

Bachelor's degree in Computer Science

Tools

FastAPI

Kafka

PostgreSQL

Kubernetes

GitHub Actions

Job description

Proven Technical Leadership: 8+ years of experience architecting and operating production backend or distributed systems, including at least 2+ years acting as a Technical Lead.

Educational Foundation: Bachelor’s degree in Computer Science, a related technical field, or equivalent practical experience.

Production AI Tracking: Hands‑on experience successfully deploying LLM‑powered applications or agents to production, with a strong understanding of handling edge cases, failures, and mitigation.

Framework & Tooling Engineering: Advanced Python proficiency combined with a track record of designing robust frameworks, developer SDKs, or platform APIs tailored for other engineering teams.

Foundational Engineering & Communication

• Distributed Systems Mastery: Deep understanding of distributed system fundamentals, including API design, asynchronous/event‑driven processing, data streaming, caching strategies, rate limiting, and fault‑tolerant architectures.
• Technical Communication: Excellent written communication skills, with proven ability to author comprehensive system design documents that articulate technical trade‑offs clearly to both engineering teams and executive leadership.

Technical Capabilities & Domain Expertise:

We are looking for a well‑rounded technical leader with broad exposure across these domains and deep expertise in a few key areas:

• Agent Development: Architecting tool/function calling, multi‑step and multi‑agent orchestration, task delegation, durable execution, and human‑in‑the‑loop approval workflows.
• Agent Interoperability: Designing open protocols to expose agents and tools seamlessly to other applications and external agents.
• AI User Experiences: Engineering streaming agent outputs to web clients and building generative or dynamic user interfaces.

Context Engineering & Lifecycle Management

• Context Engineering: Implementing Retrieval‑Augmented Generation (RAG), vector/hybrid search, short- and long‑term memory systems, automatic summarization, and routing across extensive tool catalogs.
• LLM Evaluation & Observability: Building evaluation datasets and rubrics, leveraging LLM‑as‑a‑judge, running regression testing in CI, and establishing distributed tracing, logging, metrics, SLOs, and online production evaluation.

Enterprise Platform, Security & Governance

• LLM Infrastructure: Managing cross‑provider model routing, token consumption, cost optimization, response caching, and latency reduction.
• AI Safety & Governance: Deploying prompt‑injection defenses, PII detection/redaction, guardrails, policy engines, and sandboxed code execution environments.
• Core & Multi‑Tenant Security: Designing authentication/authorization, service‑to‑service identity, least‑privilege access controls, strict tenant isolation, and detailed audit logging.
• Cloud‑Native Infrastructure: Operating containers, container orchestration, robust CI/CD pipelines, and infrastructure as code (IaC).

Core Tech Stack & Toolkit

Expertise in Python is required. For the rest of the stack, we look for hands‑on exposure to most of these technologies:

• Languages: Expert‑level Python; solid working knowledge of SQL. Java experience is a valuable asset.
• Backend APIs & Streaming: High proficiency in FastAPI, asyncio, and Pydantic; practical experience with REST, gRPC, Server‑Sent Events (SSE), WebSockets, and event streaming via Kafka or similar.

AI, Agentic Frameworks & Protocols

• Agent Frameworks & Protocols: Hands‑on experience with LangGraph and LangChain, or comparable frameworks (e.g., OpenAI Agents SDK, Google ADK, Pydantic AI) along with agent protocols like MCP, A2A, and A2UI.
• Models, Providers & Gateways: Interfacing with enterprise APIs (OpenAI, Anthropic, Google Gemini, Amazon Bedrock, Azure OpenAI), serving open‑weight models via vLLM, and managing LLM gateways like LiteLLM.

Data, Evaluation & Observability

• Data & Retrieval Engines: PostgreSQL with pgvector; vector and search engines (OpenSearch, Elasticsearch, Qdrant); Redis; and embedding/reranking models.
• Evaluation & Telemetry: OpenTelemetry (including GenAI semantic conventions); LLM tracing/eval platforms (LangSmith, Ragas, DeepEval); and standard metrics stacks (Prometheus, Grafana).

Security, Infrastructure & Frontend

• Security & Governance Tools: OAuth 2.0, OIDC, JWT, and workload identities (SPIFFE/SPIRE); policy‑as‑code (Open Policy Agent, Cedar); secrets management (HashiCorp Vault); and guardrail frameworks (NeMo Guardrails, Llama Guard, Presidio).
• Cloud‑Native & Sandboxing: Docker, Kubernetes, Helm, Terraform; experience with at least one major cloud provider (AWS, Azure, GCP); GitHub Actions or Jenkins; and secure sandbox runtimes (gVisor, Firecracker).
• Frontend Engineering: React; streaming chat interfaces; and generative UI frameworks or specifications (Vercel AI SDK, A2UI, MCP‑UI).

Preferred Qualifications:

• Platform Scale & Adoption: Proven experience building internal platforms, developer SDKs, or frameworks that were successfully adopted across multiple engineering teams.
• Enterprise AI Experience: Track record of deploying, running, and managing production AI systems tailored for external enterprise customers.
• Open Source & Ecosystem Impact: Active contributions to open‑source agentic frameworks or emerging agent protocols.