Applied AI/ML Engineer - Production LLM Systems

Certum Legal Solutions; Oregon; Hybrid; USD 175,000 - 250,000; Full time

Benefits offered by this job

Bonuses (10-20%)

Medical Insurance

Dental Insurance

Vision Insurance

Unlimited PTO

401(k) with matching

Remote or Hybrid

Job summary

Certum Legal Solutions is seeking an Applied AI/ML Engineer to own the intelligence layer of our platform, building production-grade LLM systems in a HIPAA-regulated environment. You will collaborate with a backend team to integrate AI capabilities and scale workflows across document extraction, agent orchestration, and QA layers.

The role requires 5+ years in software engineering with 2-3 years in production LLM apps, strong Python, and experience with LangChain, LlamaIndex, and LLM APIs.

Qualifications

• 5+ years of software engineering experience with at least 2-3 years focused on production LLM application development.
• Expert Python; strong API integration skills; experience deploying AI systems at production scale
• Deep familiarity with engineering reliable systems around Claude, GPT, or equivalent LLM APIs, not just calling them
• Experience with agent frameworks (LangChain, LlamaIndex, or custom orchestration)
• RAG architecture: vector databases, embedding models, retrieval optimization
• Prompt engineering discipline: systematic evaluation, regression testing, version control for prompts
• Experience handling sensitive or regulated data (HIPAA, FERPA, or similar)

Responsibilities

• Own all AI/ML system design and implementation on the platform
• Design and maintain prompt engineering infrastructure: versioned prompts, A/B evaluation, and regression testing
• Build and iterate on structured extraction pipelines for complex, regulated documents, from initial parsing through multi-step reasoning and output validation
• Design the agent orchestration framework powering task agents, QA agents, and operational agents
• Establish AI quality standards: confidence thresholds, human escalation triggers, output validation
• Partner with the Lead Backend Engineer on model abstraction layer design and API integration
• Partner with domain experts on extraction schema design and workflow logic

Skills

Production LLM applications

Python

API integration

Claude / GPT APIs

LangChain

LlamaIndex

RAG architecture

Prompt engineering

HIPAA / regulated data

Tools

LangChain

LlamaIndex

LLM APIs

Job description

Certum Legal Solutions is seeking an Applied AI/ML Engineer to own the intelligence layer of our platform, building production-grade LLM systems in a HIPAA-regulated environment. You will collaborate with a backend team to integrate AI capabilities and scale workflows across document extraction, agent orchestration, and QA layers.

The role requires 5+ years in software engineering with 2-3 years in production LLM apps, strong Python, and experience with LangChain, LlamaIndex, and LLM APIs.