Senior AI Engineer — End-to-End LLM Optimization

LawPro.ai; Georgia; On-site; USD 140,000 - 200,000; Full time

Job summary

LawPro.ai is seeking an experienced AI Engineer to own evaluation, selection, and optimization of large language models powering its data insights platform. You will ensure AI systems remain accurate, cost-effective, and resilient while transitioning to new models and technologies in a fast-moving landscape.

This role blends AI research with production engineering; you will build evaluation frameworks, benchmark models, and implement production-ready changes with end-to-end ownership,

Qualifications

• 5+ years of AI/ML engineering experience in production environments.
• Experience deploying models to cloud infrastructure (AWS or GCP).
• Hands-on development of RAG solutions.
• Experience with embedding models and vector databases.
• Experience building agentic workflows and EvalOps.
• Strong understanding of LLMs, costs, performance, and reliability.
• Proven ability to ship production-grade LLM orchestration pipelines.
• Excellent communication with cross-functional teams.
• Bonus: medical/legal document processing or retrieval-augmented generation.

Responsibilities

• Design and operate ongoing LLM evaluation for accuracy, speed, and cost.
• Develop robust evaluation frameworks (Evals) and EvalOps culture.
• Plan and execute seamless model transitions with production integration.
• Optimize AI pipelines for document understanding and case generation.
• Collaborate with product and GTM to communicate findings and implement changes.
• Own end-to-end shipping of model changes into production with tests and rollout.
• Implement monitoring and observability for model performance and cost in production.
• Document evaluation methodologies and runbooks.

Skills

AI/ML engineering

LLM evaluation

RAG solutions

Embedding models

Production deployment

Education

Bachelor's degree in CS/CE

Tools

AWS

GCP

Vector databases

EvalOps

Job description

LawPro.ai is seeking an experienced AI Engineer to own evaluation, selection, and optimization of large language models powering its data insights platform. You will ensure AI systems remain accurate, cost-effective, and resilient while transitioning to new models and technologies in a fast-moving landscape.

This role blends AI research with production engineering; you will build evaluation frameworks, benchmark models, and implement production-ready changes with end-to-end ownership,