Applied AI/ML Engineer

Certum Legal Solutions; Myrtle Point (OR); Remote; USD 175,000 - 250,000; Full time

Benefits offered by this job

Medical, dental, and vision insurance

Unlimited PTO

401(k) with employer matching

Remote or hybrid work schedule

Job summary

Certum Legal Solutions is building an AI-native platform and seeks an Applied AI/ML Engineer to own the intelligence layer, focusing on production LLM systems that operate at scale in a HIPAA-regulated environment, collaborating with the Lead Backend Engineer to integrate capabilities from day one.

You will design extraction pipelines, agent orchestration, RAG layers, and QA workflows, while ensuring security, reliability, and scalability across high-volume legal data.

Qualifications

• 5+ years of software engineering with 2–3 years in production LLM app development
• Expert Python; strong API integration; production-scale AI systems
• Deep familiarity with Claude, GPT, or equivalent LLM APIs beyond basic calls
• Experience with agent frameworks (LangChain, LlamaIndex, or custom orchestration)
• RAG architecture: vector databases, embeddings, retrieval optimization
• Prompt engineering discipline: evaluation, regression testing, version control for prompts
• Experience handling sensitive or regulated data (HIPAA, FERPA, or similar)

Responsibilities

• Own all AI/ML system design and implementation on the platform
• Design and maintain prompt engineering infrastructure: versioned prompts, A/B evaluation, regression tests
• Build and iterate on structured extraction pipelines for complex documents from parsing to multi-step reasoning
• Design the agent orchestration framework powering task, QA, and operational agents
• Establish AI quality standards: confidence thresholds, human escalation triggers, output validation
• Partner with Backend Engineer on model abstraction layer design and API integration
• Collaborate with domain experts on extraction schema design and workflow logic

Skills

Python

API integration

LLM APIs

Agent frameworks

RAG architecture

Prompt engineering

HIPAA data handling

Tools

LangChain

LlamaIndex

custom orchestration

vector databases

embedding models

retrieval optimization

Job description

Applied AI/ML Engineer

Remote, US

Certum Legal Solutions (CLS) is a pioneering legal managed services organization (MSO) that supports mass-tort and single-event personal injury law firms. We provide the operational infrastructure, technology, and back-office solutions that allow the firms on our platform to grow and scale while focusing on delivering industry-leading results for their clients.

We are building a proprietary, AI-native system that will transform every dimension of how CLS and its platform firms operate. This system encompasses AI-powered agents to manage end-to-end workflows throughout the lifecycle of mass-tort and single-event personal-injury litigation matters, including back-office support functions such as HR, accounting and finance, and IT. This is a greenfield build on a modern stack, designed from Day 1 to be extensible and, eventually, productized for the broader legal services market. We are assembling a high-caliber team to build something that does not exist in this form anywhere in the legal industry. If you want to see your work directly transform a real business and believe AI is fundamentally reshaping professional services, this is the team to join.

ABOUT THE ROLE

The Applied AI/ML Engineer owns the intelligence layer of Project Interstellar: the infrastructure that extracts structured data for robust document intelligence, powers conversational voice agents, orchestrates multi-step task agents, and handles various other operational workflows. This is not a research role. It is an applied AI/ML engineering role focused on building production LLM systems that perform reliably at scale in a HIPAA-regulated environment. You will work directly with the Lead Backend Engineer to integrate AI capabilities into the platform architecture from the ground up.

WHAT YOU WILL BUILD

• Production-grade structured data extraction pipeline for complex, multi-page domain-specific documents: entity extraction, classification, confidence scoring, temporal sequencing, and calculated output generation
• Model abstraction layer with task-based routing, enabling cost-optimized inference across workloads of varying complexity (from high-volume classification to deep multi-step reasoning)
• Agent orchestration framework and tool registry enabling task-executing agents to receive natural language instructions, plan multi-step executions, and call platform services
• Quality assurance agent layer: automated rubric-based verification of upstream agent outputs against defined accuracy and completeness standards
• Training and evaluation agent system: interactive scenario simulation and competency assessment for platform operators
• RAG layer over domain-specific internal knowledge bases, enabling intelligent lookup and context injection for operational agents

RESPONSIBILITIES

• Own all AI/ML system design and implementation on the platform
• Design and maintain prompt engineering infrastructure: versioned prompts, A/B evaluation, and regression testing
• Build and iterate on structured extraction pipelines for complex, regulated documents, from initial parsing through multi-step reasoning and output validation
• Design the agent orchestration framework powering task agents, QA agents, and operational agents
• Establish AI quality standards: confidence thresholds, human escalation triggers, output validation
• Partner with the Lead Backend Engineer on model abstraction layer design and API integration
• Partner with domain experts on extraction schema design and workflow logic

REQUIRED SKILLS

• 5+ years of software engineering experience with at least 2–3 years focused on production LLM application development
• Expert Python; strong API integration skills; experience deploying AI systems at production scale
• Deep familiarity with engineering reliable systems around Claude, GPT, or equivalent LLM APIs, not just calling them
• Experience with agent frameworks (LangChain, LlamaIndex, or custom orchestration)
• RAG architecture: vector databases, embedding models, retrieval optimization
• Prompt engineering discipline: systematic evaluation, regression testing, version control for prompts
• Experience handling sensitive or regulated data (HIPAA, FERPA, or similar)

PREFERRED QUALIFICATIONS

• Document and text extraction from unstructured sources (OCR post-processing, information extraction from clinical or legal documents)
• Conversational AI or voice AI systems (NLU, dialog management, conversation evaluation)
• Legal or healthcare AI applications
• Multi-agent orchestration patterns
• Experience evaluating and scoring LLM outputs at scale

COMPENSATION

• Base Salary: $175,000 – $250,000 per year, based on experience and skillset
• Deliverables and KPI-based performance bonuses (10-20% of Base Salary)
• Medical, dental, and vision insurance
• Unlimited PTO plus company holidays
• 401(k) eligibility upon start with employer matching
• Remote or hybrid work schedule

EQUAL EMPLOYMENT OPPORTUNITY STATEMENT

Certum Legal Solutions is an