Forward Deployment Engineer

Newnovation

San Francisco (CA)

On-site

USD 200,000 - 225,000

Full time

4 days ago
Be an early applicant

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Job summary

Newnovation seeks a senior AI/ML engineer to own the intelligence layer for multiple programs, focusing on model quality, RAG accuracy, prompt engineering, and safety guardrails. You will architect RAG evaluation frameworks, tune vector retrieval, and optimize multi-step LLM call chains for latency in production environments.

The role requires 10+ years in software/AI, with 4+ years in LLM production, and a track record of shipping production-ready AI features with measurable metrics.

Qualifications

  • 10+ years of relevant professional experience in software engineering, application development, or AI/ML engineering.
  • 4+ years of software engineering with at least 2 years focused on LLM application development in production.
  • Shipped an LLM-powered feature or product to production with measurable user metrics.
  • Owned an AI safety/guardrails implementation for a customer-facing product.
  • Built RAG evaluation pipelines and used them for go/no-go release decisions.
  • Proficient in latency optimization of multi-step LLM call chains.

Responsibilities

  • Own the intelligence layer across programs, ensuring model quality, RAG accuracy, and prompt engineering.
  • Maintain AI safety guardrails across applications and ensure regulatory compliance.
  • Design and implement RAG evaluation and retrieval feedback loops.
  • Optimize end-to-end LLM call chains for latency and throughput.
  • Collaborate with product teams to deploy production-ready AI features.

Skills

LLM prompt engineering
Multi-turn conversation design
Model selection & benchmarking
Vector similarity search tuning
RAG evaluation frameworks
Hybrid search
Prompt injection detection
Jailbreak testing
Content safety classifier integration
Hallucination detection
Topical control
Automated evaluation pipelines
Latency profiling
Embedding model operations

Tools

RAGAS
TruLens

Job description

Location: Bay area, CA

Duration: FTE

Range: $200k to $225k Per Annum

  • Will work on the intelligence layer for multiple programs — owns all model quality, RAG accuracy, prompt engineering, and AI safety across applications
  • Socratic tutor persona, adaptive learning recommendation engine, multi-modal AI (text and voice), RAG evaluation framework, and feedback loop into retrieval
  • Production-grade AI quality from launch — this is not a research or prototyping role; accuracy thresholds, latency requirements, and safety guardrails must pass InfoSec adversarial testing before Release 1
Required Skills
Experience
  • 10+ years of relevant professional experience in software engineering, application development, or AI/ML engineering.
  • 4+ years of software engineering with at least 2 years focused on LLM application development in production. Demonstrated experience delivering and maintaining LLM-powered applications in production environments, with measurable user adoption, accuracy, and quality metrics.
  • Has shipped an LLM-powered feature or product to production where real users depend on the accuracy and the engineer owns the quality metrics
  • Has owned an AI safety or guardrails implementation for a customer-facing product — not just added an off-the-shelf filter; designed and tested the safety layer
  • Has built RAG evaluation pipelines and used them to make go/no-go release decisions — accuracy gating is part of the workflow.
  • Has profiled and optimized a multi-step LLM call chain for latency
Must Have
LLM Application Development
  • LLM prompt engineering — system prompts, few-shot examples, chain-of-thought, instruction following · Expert
  • Multi-turn conversation design — context window management, conversation summarization, session memory · Advanced ·
  • Model selection and benchmarking — matching model size to task; balancing latency, cost, and accuracy · Advanced ·
  • Vector similarity search tuning — index parameters, similarity thresholds, retrieval depth · Advanced ·Reranking — cross-encoder rerankers, relevance scoring · Advanced ·
  • RAG evaluation frameworks — RAGAS, TruLens, or equivalent; automated eval pipelines · Advanced ·
  • Hybrid search — combining dense vector retrieval with BM25 or keyword search · Proficient ·
AI Safety & Guardrails
  • Prompt injection detection and mitigation · Advanced ·
  • Jailbreak testing and red-teaming LLM systems · Advanced
  • Content safety classifier integration · Advanced
  • Hallucination detection and mitigation strategies · Advanced
  • Topical control — enforcing scope boundaries on LLM responses · Advanced ·
Evaluation & Production Quality
  • Automated evaluation pipeline design — test set curation, metric selection, regression detection · Advanced ·
  • A/B evaluation methodology for prompt and model changes · Proficient
  • Latency profiling for LLM call chains — identifying bottlenecks across multi-step pipelines · Proficient ·
Development
  • Embedding model operations — model selection, batch embedding, index updates · Advanced ·
Nice to Have
  • Adaptive learning systems or personalization engine experience
  • Knowledge graph integration with RAG
  • Multi-agent orchestration patterns
  • ServiceNow API integration
  • Prior experience building AI products on NVIDIA infrastructure
Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

AI Engineer
AI Engineer

Minfy • McLean (VA)

Hybrid
USD 150,000 - 230,000
Senior LLMOps Engineer
Senior LLMOps Engineer

UNAVAILABLE • McLean (VA)

On-site
USD 180,000 - 240,000
AI Engineer - GA
AI Engineer - GA

LawPro.ai • Georgia

On-site
USD 140,000 - 200,000
Principal Engineer - Context Engineering & LLM Optimization
Principal Engineer - Context Engineering & LLM Optimization

HITEC • Charlotte (NC)

On-site
USD 130,000 - 160,000
AI Engineer (Remote)
AI Engineer (Remote)

7Seventy Recruiting • United States

On-site
USD 150,000 - 175,000
LLM / GenAI Engineer
LLM / GenAI Engineer

Evlo AI • Washington

On-site
USD 180,000 - 260,000
LLM / GenAI Engineer
LLM / GenAI Engineer

Evlo AI • Miami (FL)

On-site
USD 120,000 - 180,000
AI Engineer, Multimodal LLMs
AI Engineer, Multimodal LLMs

Latitude • San Francisco (CA)

On-site
USD 140,000 - 200,000
AI Engineer - NC
AI Engineer - NC

LawPro.ai • North Carolina

On-site
USD 140,000 - 190,000
LLM / GenAI Engineer
LLM / GenAI Engineer

Evlo AI • Minneapolis (MN)

On-site
USD 140,000 - 190,000