Applied AI/ML Engineer

Certum Legal Solutions; United States; Hybrid; USD 175,000 - 250,000; Full time

Benefits offered by this job

Medical insurance

Dental insurance

Vision insurance

Unlimited PTO

401(k) with company match

Remote or hybrid work

Job summary

Certum Legal Solutions seeks an Applied AI/ML Engineer to own the intelligence layer of Project Interstellar, building production LLM systems in a HIPAA-regulated environment, collaborating with the Lead Backend Engineer to embed AI capabilities across the platform.

You will design extraction pipelines, task orchestration, and QA/training agents, advancing a greenfield system aimed at transforming regulatory document processing and law firm workflows through scalable AI-driven automation.

Qualifications

• 5+ years in software engineering with 2–3 years in production LLM apps.
• Expert Python and strong API integration.
• Experience deploying AI systems at production scale in regulated environments.
• Familiarity with Claude, GPT or similar LLM APIs beyond simple calls.

Responsibilities

• Own AI/ML system design and implementation on the platform.
• Design and maintain prompt engineering infrastructure with version control.
• Build structured extraction pipelines for regulated documents.
• Design the agent orchestration framework powering multiple agent types.
• Establish AI quality standards with confidence thresholds and escalation triggers.
• Collaborate with backend engineers and domain experts on schemas and workflows.

Skills

Python programming

API integration

LLM production deployment

LangChain experience

RAG architecture

Prompt engineering

HIPAA data handling

Tools

LangChain

LlamaIndex

Vector databases

Embedding models

Custom orchestration framework

Job description

Certum Legal Solutions (CLS) is a pioneering legal managed services organization (MSO) that supports mass-tort and single-event personal injury law firms. We provide the operational infrastructure, technology, and back-office solutions that allow the firms on our platform to grow and scale while focusing on delivering industry-leading results for their clients.

We are building a proprietary, AI-native system that will transform every dimension of how CLS and its platform firms operate. This system encompasses AI-powered agents to manage end-to-end workflows throughout the lifecycle of mass-tort and single-event personal-injury litigation matters, including back-office support functions such as HR, accounting and finance, and IT. This is a greenfield build on a modern stack, designed from Day 1 to be extensible and, eventually, productized for the broader legal services market. We are assembling a high-caliber team to build something that does not exist in this form anywhere in the legal industry. If you want to see your work directly transform a real business and believe AI is fundamentally reshaping professional services, this is the team to join.

ABOUT THE ROLE

The Applied AI/ML Engineer owns the intelligence layer of Project Interstellar: the infrastructure that extracts structured data for robust document intelligence, powers conversational voice agents, orchestrates multi-step task agents, and handles various other operational workflows. This is not a research role. It is an applied AI/ML engineering role focused on building production LLM systems that perform reliably at scale in a HIPAA-regulated environment. You will work directly with the Lead Backend Engineer to integrate AI capabilities into the platform architecture from the ground up.

WHAT YOU WILL BUILD

• Production-grade structured data extraction pipeline for complex, multi-page domain-specific documents: entity extraction, classification, confidence scoring, temporal sequencing, and calculated output generation
• Model abstraction layer with task-based routing, enabling cost-optimized inference across workloads of varying complexity (from high-volume classification to deep multi-step reasoning)
• Agent orchestration framework and tool registry enabling task-executing agents to receive natural language instructions, plan multi-step executions, and call platform services
• Quality assurance agent layer: automated rubric-based verification of upstream agent outputs against defined accuracy and completeness standards
• Training and evaluation agent system: interactive scenario simulation and competency assessment for platform operators
• RAG layer over domain-specific internal knowledge bases, enabling intelligent lookup and context injection for operational agents

RESPONSIBILITIES

• Own all AI/ML system design and implementation on the platform
• Design and maintain prompt engineering infrastructure: versioned prompts, A/B evaluation, and regression testing
• Build and iterate on structured extraction pipelines for complex, regulated documents, from initial parsing through multi-step reasoning and output validation
• Design the agent orchestration framework powering task agents, QA agents, and operational agents
• Establish AI quality standards: confidence thresholds, human escalation triggers, output validation
• Partner with the Lead Backend Engineer on model abstraction layer design and API integration
• Partner with domain experts on extraction schema design and workflow logic

REQUIRED SKILLS

• 5+ years of software engineering experience with at least 2–3 years focused on production LLM application development
• Expert Python; strong API integration skills; experience deploying AI systems at production scale
• Deep familiarity with engineering reliable systems around Claude, GPT, or equivalent LLM APIs, not just calling them
• Experience with agent frameworks (LangChain, LlamaIndex, or custom orchestration)
• RAG architecture: vector databases, embedding models, retrieval optimization
• Prompt engineering discipline: systematic evaluation, regression testing, version control for prompts
• Experience handling sensitive or regulated data (HIPAA, FERPA, or similar)

PREFERRED QUALIFICATIONS

• Document and text extraction from unstructured sources (OCR post-processing, information extraction from clinical or legal documents)
• Conversational AI or voice AI systems (NLU, dialog management, conversation evaluation)
• Legal or healthcare AI applications
• Multi-agent orchestration patterns
• Experience evaluating and scoring LLM outputs at scale

COMPENSATION

Base Salary: $175,000 – $250,000 per year, based on experience and skillset

• Deliverables and KPI-based performance bonuses (10-20% of Base Salary)
• Medical, dental, and vision insurance
• Unlimited PTO plus company holidays
• 401(k) eligibility upon start with employer matching
• Remote or hybrid work schedule

EQUAL EMPLOYMENT OPPORTUNITY STATEMENT

Certum Legal Solutions is an Equal Opportunity Employer. All qualified applicants will receive consideration for employment without regard to race, color, creed, religion, sex, national origin, ancestry, age, marital status, civil union status, domestic partnership status, disability, atypical hereditary cellular or blood trait, genetic information, veteran status, affectional or sexual orientation, gender identity or expression, or any other characteristic protected by applicable law.