Healthcare AI/LLM Engineer (Remote)

Codoxo; Atlanta (GA); Hybrid; USD 120,000 - 190,000; Full time

Benefits offered by this job

Health insurance

Unlimited PTO

Annual home office stipend

401K match

Job summary

Codoxo is seeking an AI/LLM Engineer to design, ship, and operate LLM-powered features that accelerate claim audits and SIU investigations. You’ll handle retrieval, extraction, and summarization over sensitive documents, and build evaluation systems to prove effectiveness.

Our stack is Python and Django on AWS, using Bedrock for inference, OpenSearch for vector search, and Celery for processing. You will own features end to end and influence future multi-agent orchestration.

Qualifications

• 3+ years of professional Python experience.
• 2+ years building with LLMs in production.
• Hands-on RAG experience: embeddings, vector or hybrid search.
• Real practice around evaluation: know your eval set and metrics.
• Working knowledge of AWS serverless: Lambda, S3, API Gateway; Bedrock experience is a plus.
• Comfort with PHI or regulated data.

Responsibilities

• Build and ship LLM features for investigator workflows (QA, extraction, summarization).
• Own end-to-end retrieval pipeline: ingestion, extraction, chunking, filtering, hybrid search, citations.
• Develop and maintain evaluation systems: golden sets, regression suites, rubric-based grading.
• Treat prompts as engineering artifacts: versioned, code-reviewed, documented, with eval runs.
• Design constrained schemas for structured output and handle refusals gracefully.
• Work on serverless AWS stack (Bedrock, Lambda, S3, API Gateway, Step Functions).
• Optimize cost and latency: token accounting, caching, routing tasks to model tiers.
• Instrument features for observability: versions, token counts, retry rates, document IDs.
• Collaborate with investigators, coders, and product to define eval criteria.

Skills

Python

LLMs in production

RAG and embeddings

Evaluation methodologies

AWS serverless

PHI handling

Tools

OpenSearch

Bedrock

AWS Lambda

Django

Job description

Codoxo is seeking an AI/LLM Engineer to design, ship, and operate LLM-powered features that accelerate claim audits and SIU investigations. You’ll handle retrieval, extraction, and summarization over sensitive documents, and build evaluation systems to prove effectiveness.

Our stack is Python and Django on AWS, using Bedrock for inference, OpenSearch for vector search, and Celery for processing. You will own features end to end and influence future multi-agent orchestration.