Senior AI Engineer - Evaluation & Agent Infra (Remote)

The Wrap; United States; Hybrid; USD 160,000 - 185,000; Full time

Benefits offered by this job

Medical insurance

Dental insurance

Vision insurance

Life and Disability Insurance

Emergency Weather Support

Wellness Apps

Paid Parental Leave

Paid Time off

Job summary

Lattice is seeking an experienced AI Engineer to help shape evaluation methodologies, agent architecture, and core AI systems at scale. The role emphasizes ownership and collaboration with product and design to deliver reliable AI experiences.

You will design an end-to-end evaluation framework, build scalable agent infrastructure, and contribute to production-grade systems with a strong emphasis on reliability, observability, and performance.

Qualifications

• Five+ years of professional software engineering experience with production AI/ML systems.
• Hands-on with LLM-based systems: prompt engineering, RAG pipelines, agent orchestration, evaluation metrics, model fine-tuning.

Responsibilities

• Design and ship an end-to-end AI evaluation framework across offline evals, tracing, and human-in-the-loop feedback.
• Architect reusable agent infrastructure and scalable retrieval systems, with focus on build-vs-buy decisions.
• Own projects end-to-end, driving them to completion and aligning with product and design teams.

Skills

AI/ML systems

Python

LLM-based systems

Prompt engineering

RAG pipelines

Agent orchestration

Production experience

Statistics in experiments

Technical communication

Tools

LangGraph

LangSmith

Pinecone

AWS

TypeScript

Job description

Lattice is seeking an experienced AI Engineer to help shape evaluation methodologies, agent architecture, and core AI systems at scale. The role emphasizes ownership and collaboration with product and design to deliver reliable AI experiences.

You will design an end-to-end evaluation framework, build scalable agent infrastructure, and contribute to production-grade systems with a strong emphasis on reliability, observability, and performance.