Production LLM AI Engineer: RAG, Safety & Latency

Newnovation; San Francisco (CA); On-site; USD 200,000 - 225,000; Full time

Job summary

Newnovation seeks a senior AI/ML engineer to own the intelligence layer for multiple programs, focusing on model quality, RAG accuracy, prompt engineering, and safety guardrails. You will architect RAG evaluation frameworks, tune vector retrieval, and optimize multi-step LLM call chains for latency in production environments.

The role requires 10+ years in software/AI, with 4+ years in LLM production, and a track record of shipping production-ready AI features with measurable metrics.

Qualifications

• 10+ years of relevant professional experience in software engineering, application development, or AI/ML engineering.
• 4+ years of software engineering with at least 2 years focused on LLM application development in production.
• Shipped an LLM-powered feature or product to production with measurable user metrics.
• Owned an AI safety/guardrails implementation for a customer-facing product.
• Built RAG evaluation pipelines and used them for go/no-go release decisions.
• Proficient in latency optimization of multi-step LLM call chains.

Responsibilities

• Own the intelligence layer across programs, ensuring model quality, RAG accuracy, and prompt engineering.
• Maintain AI safety guardrails across applications and ensure regulatory compliance.
• Design and implement RAG evaluation and retrieval feedback loops.
• Optimize end-to-end LLM call chains for latency and throughput.
• Collaborate with product teams to deploy production-ready AI features.

Skills

LLM prompt engineering

Multi-turn conversation design

Model selection & benchmarking

Vector similarity search tuning

RAG evaluation frameworks

Hybrid search

Prompt injection detection

Jailbreak testing

Content safety classifier integration

Hallucination detection

Topical control

Automated evaluation pipelines

Latency profiling

Embedding model operations

Tools

RAGAS

TruLens

Job description

Newnovation seeks a senior AI/ML engineer to own the intelligence layer for multiple programs, focusing on model quality, RAG accuracy, prompt engineering, and safety guardrails. You will architect RAG evaluation frameworks, tune vector retrieval, and optimize multi-step LLM call chains for latency in production environments.

The role requires 10+ years in software/AI, with 4+ years in LLM production, and a track record of shipping production-ready AI features with measurable metrics.