Senior AI Engineer - Production LLM EvalOps

LawPro.ai; Town of Texas (WI); On-site; USD 140,000 - 210,000; Full time

Job summary

LawPro.ai is seeking an experienced AI Engineer to own evaluation, selection, and optimization of LLMs powering its data insights platform. You will drive end-to-end model transitions, benchmark new approaches, and implement production-ready solutions with a bias for quality and cost efficiency.

You will build evaluation frameworks, collaborate with product and operations, and ship changes to production, while maintaining documentation and monitoring for drift and performance.

Qualifications

• 5+ years AI/ML engineering in production.

Responsibilities

• Design and operate ongoing evaluation of new LLMs across accuracy, speed, and cost with proactive benchmarking.

Skills

LLM Evaluation

RAG Solutions

Embeddings

Vector DBs

EvalOps

Production Deploy

Communication

Cloud Infra

Documentation

Tools

AWS

GCP

Job description

LawPro.ai is seeking an experienced AI Engineer to own evaluation, selection, and optimization of LLMs powering its data insights platform. You will drive end-to-end model transitions, benchmark new approaches, and implement production-ready solutions with a bias for quality and cost efficiency.

You will build evaluation frameworks, collaborate with product and operations, and ship changes to production, while maintaining documentation and monitoring for drift and performance.