Remote Senior AI Engineer - LLM Eval & Production

LawPro.ai; United States; Remote; USD 150,000 - 170,000; Full time

Benefits offered by this job

100% remote in US

Competitive salary

Equity participation

Unlimited PTO

Health, dental, vision benefits

Growth opportunities

Job summary

LawPro.ai seeks an experienced AI Engineer to own evaluation, selection, and optimization of LLMs powering our legal analytics platform. You will ensure accuracy, cost-effectiveness, and resilience while transitioning to new models and technologies.

The role blends AI research with production engineering and requires systematic, scalable approaches. You will help design evaluation frameworks, benchmark models, and lead end-to-end deployment with a bias toward practical, production-ready

Qualifications

• 5+ years of AI/ML engineering experience in production environments.
• Experience evaluating, fine-tuning, and deploying LLMs.
• Hands-on with embedding models and vector databases.
• Ability to design evaluation frameworks for high-stakes domains (legal/medical).
• Strong communication skills with technical and non-technical stakeholders.

Responsibilities

• Design and operate a systematic process to evaluate new LLMs across accuracy, speed, and cost.
• Build and maintain evaluation frameworks and measure output quality, reducing hallucinations.
• Own model transitions, deprecations, and production integrations of new models.
• Optimize AI pipelines for document understanding, summarization, and chronology generation.
• Lead cross-functional collaboration and ensure clean handoffs from discovery to production.
• Implement monitoring and observability for model performance and cost.

Skills

AI/ML engineering

RAG solutions

Embedding models

Vector databases

EvalOps

Production deployment

Tools

AWS

GCP

Job description

LawPro.ai seeks an experienced AI Engineer to own evaluation, selection, and optimization of LLMs powering our legal analytics platform. You will ensure accuracy, cost-effectiveness, and resilience while transitioning to new models and technologies.

The role blends AI research with production engineering and requires systematic, scalable approaches. You will help design evaluation frameworks, benchmark models, and lead end-to-end deployment with a bias toward practical, production-ready