Applied AI/ML Engineer

Certum Legal Solutions; United States; Hybrid; USD 175,000 - 250,000; Full time

Benefits offered by this job

Bonus based on KPIs

Insurance (medical, dental, vision)

401(k) with company matching

Remote or hybrid work

Job summary

Certum Legal Solutions (CLS) is building an AI-native platform to transform mass-tort and personal-injury litigation workflows. The Applied AI/ML Engineer owns the intelligence layer and will help integrate AI capabilities from the ground up in a HIPAA-regulated environment.

You will design data extraction pipelines, agent orchestration, and QA/validation frameworks, partnering with domain experts to shape extraction schema and workflow logic for scalable, compliant platform operations.

Qualifications

• ,
• 5+ years software engineering with 2–3 years focused on production LLM application development.
• Expert Python; strong API integration skills; experience deploying AI systems at production scale.
• Deep familiarity with engineering reliable systems around Claude, GPT, or equivalent LLM APIs, not just calling them.

Responsibilities

• Own AI/ML system design and implementation on the platform.
• Design and maintain prompt engineering infrastructure: versioned prompts, A/B evaluation, and regression testing.
• Build and iterate on structured extraction pipelines for complex, regulated documents, from initial parsing through multi-step reasoning and output validation.
• Design the agent orchestration framework powering task agents, QA agents, and operational agents.
• Establish AI quality standards: confidence thresholds, human escalation triggers, output validation.
• Partner with the Lead Backend Engineer on model abstraction layer design and API integration.
• Partner with domain experts on extraction schema design and workflow logic.

Skills

Python

API integration

Production LLMs

LangChain

LlamaIndex

RAG

Prompt engineering

HIPAA data handling

Tools

Claude API

GPT API

Vector databases

Job description

Certum Legal Solutions (CLS) is a pioneering legal managed services organization (MSO) that supports mass-tort and single-event personal injury law firms. We provide the operational infrastructure, technology, and back-office solutions that allow the firms on our platform to grow and scale while focusing on delivering industry-leading results for their clients.

We are building a proprietary, AI-native system that will transform every dimension of how CLS and its platform firms operate. This system encompasses AI-powered agents to manage end-to-end workflows throughout the lifecycle of mass-tort and single-event personal-injury litigation matters, including back-office support functions such as HR, accounting and finance, and IT. This is a greenfield build on a modern stack, designed from Day 1 to be extensible and, eventually, productized for the broader legal services market.

We are assembling a high-caliber team to build something that does not exist in this form anywhere in the legal industry. If you want to see your work directly transform a real business and believe AI is fundamentally reshaping professional services, this is the team to join.

ABOUT THE ROLE

The Applied AI/ML Engineer owns the intelligence layer of Project Interstellar: the infrastructure that extracts structured data for robust document intelligence, powers conversational voice agents, orchestrates multi-step task agents, and handles various other operational workflows. This is not a research role. It is an applied AI/ML engineering role focused on building production LLM systems that perform reliably at scale in a HIPAA-regulated environment. You will work directly with the Lead Backend Engineer to integrate AI capabilities into the platform architecture from the ground up.

WHAT YOU WILL BUILD

• Production-grade structured data extraction pipeline for complex, multi-page domain-specific documents: entity extraction, classification, confidence scoring, temporal sequencing, and calculated output generation
• Model abstraction layer with task-based routing, enabling cost-optimized inference across workloads of varying complexity (from high-volume classification to deep multi-step reasoning)
• Agent orchestration framework and tool registry enabling task-executing agents to receive natural language instructions, plan multi-step executions, and call platform services
• Quality assurance agent layer: automated rubric-based verification of upstream agent outputs against defined accuracy and completeness standards
• Training and evaluation agent system: interactive scenario simulation and competency assessment for platform operators
• RAG layer over domain-specific internal knowledge bases, enabling intelligent lookup and context injection for operational agents

RESPONSIBILITIES

• Own all AI/ML system design and implementation on the platform
• Design and maintain prompt engineering infrastructure: versioned prompts, A/B evaluation, and regression testing
• Build and iterate on structured extraction pipelines for complex, regulated documents, from initial parsing through multi-step reasoning and output validation
• Design the agent orchestration framework powering task agents, QA agents, and operational agents
• Establish AI quality standards: confidence thresholds, human escalation triggers, output validation
• Partner with the Lead Backend Engineer on model abstraction layer design and API integration
• Partner with domain experts on extraction schema design and workflow logic

REQUIRED SKILLS

• 5+ years of software engineering experience with at least 2–3 years focused on production LLM application development
• Expert Python; strong API integration skills; experience deploying AI systems at production scale
• Deep familiarity with engineering reliable systems around Claude, GPT, or equivalent LLM APIs, not just calling them
• Experience with agent frameworks (LangChain, LlamaIndex, or custom orchestration)
• RAG architecture: vector databases, embedding models, retrieval optimization
• Prompt engineering discipline: systematic evaluation, regression testing, version control for prompts
• Experience handling sensitive or regulated data (HIPAA, FERPA, or similar)

PREFERRED QUALIFICATIONS

• Document and text extraction from unstructured sources (OCR post-processing, information extraction from clinical or legal documents)
• Conversational AI or voice AI systems (NLU, dialog management, conversation evaluation)
• Legal or healthcare AI applications
• Experience evaluating and scoring LLM outputs at scale

COMPENSATION

Base Salary: $175,000 – $250,000 per year, based on experience and skillset

• Deliverables and KPI-based performance bonuses (10-20% of Base Salary)
• Medical, dental, and vision insurance
• 401(k) eligibility upon start with employer matching
• Remote or hybrid work schedule

EQUAL EMPLOYMENT OPPORTUNITY STATEMENT

Certum Legal Solutions is an Equal Opportunity Employer. All qualified applicants will receive consideration for employment without regard to race, color, creed, religion, sex, national origin, ancestry, age, marital status, civil union status, domestic partnership status, disability, atypical hereditary cellular or blood trait, genetic information, veteran status, affectional or sexual orientation, gender identity or expression, or any other characteristic protected by applicable law.