Lead AI Systems Architect for Production LLM & Retrieval

Harnham; San Francisco (CA); On-site; USD 180,000 - 240,000; Full time

Job summary

Harnham in San Francisco seeks a senior AI systems engineer to shape how AI is architected across the company. You will own production LLM pipelines, retrieval systems, and orchestration layers, making foundational decisions on systems, evaluation, and long-term direction.

You’ll collaborate across product, platform, and domain teams to translate complex requirements into scalable, observable AI infrastructure with emphasis on reliability, security, and cost efficiency in production.

Qualifications

• 6+ years of software engineering, including 3+ years on ML/AI systems.
• Proven experience deploying production LLM systems.
• Strong background in RAG, embeddings, and vector databases.
• Experience designing evaluation systems and quantitative model improvements.
• Strong Python skills and solid software engineering fundamentals.
• Ability to make architectural decisions influencing teams and direction.
• Experience with production systems, reliability, observability and cost tradeoffs.
• Clear communication skills and cross-functional collaboration.

Responsibilities

• Design and own production AI systems end-to-end, including LLM pipelines and orchestration layers.
• Build and scale RAG systems, reranking pipelines, and vector-based search infrastructure.
• Define evaluation frameworks to measure retrieval quality and model performance.
• Analyze production behavior, identify failure modes, and drive data-driven improvements.
• Make key architectural decisions across model infrastructure and tooling.
• Partner with product, platform, and domain teams to translate requirements into scalable systems.
• Lead best practices for reliability, observability, and cost-efficient AI systems.

Skills

Production AI systems

LLM deployment

RAG & embeddings

Vector databases

Python programming

Reliability & observability

Cross-functional collaboration

Architecture decisions

Evaluation frameworks

Ownership & initiative

Job description

Harnham in San Francisco seeks a senior AI systems engineer to shape how AI is architected across the company. You will own production LLM pipelines, retrieval systems, and orchestration layers, making foundational decisions on systems, evaluation, and long-term direction.

You’ll collaborate across product, platform, and domain teams to translate complex requirements into scalable, observable AI infrastructure with emphasis on reliability, security, and cost efficiency in production.