Agentic AI/LLM Engineer, Coverage Intelligence

Bold Penguin, Inc.; Dublin (OH); Remote; USD 140,000 - 200,000; Full time

Benefits offered by this job

Annual Performance Bonus Eligible

Medical, Dental, and Vision

Flexible PTO Policy

401(k) with company match

Employee Assistance Program

Disability and Life Benefits

Job summary

Bold Penguin, Inc. is seeking an Agentic AI / LLM Engineer to design and build AI systems for converting complex insurance documents into reliable coverage intelligence. You will own end-to-end problems from understanding unstructured documents to building agents that reason about coverage.

You will ground outputs in source documents, collaborate with domain experts and engineers, and drive accuracy through rigorous evaluation and iterative improvement.

Qualifications

• 5+ years' experience in software engineering or applied machine learning.
• 2+ years building LLM applications in production.
• Strong Python skills with typed schemas (e.g., Pydantic).
• Hands-on experience extracting structured data from complex documents (PDF parsing, OCR).
• Experience with agentic patterns: tool use, multi-step workflows, human-in-the-loop.

Responsibilities

• Design, build and own production AI systems for document understanding and coverage analysis.
• Build agentic workflows combining models, tools and human review.
• Ground outputs in source documents and domain knowledge for traceability.
• Drive accuracy via evaluation, experimentation and error analysis.

Skills

LLM applications in production

Python

PDF parsing/ OCR

Agentic workflows

Retrieval-augmented generation

Evaluation/metrics

Communication

Autonomy in work

LangGraph

LLM observability tooling

Tools

LangGraph

LLM observability tooling

Job description

• Other Compensation Annual Performance Bonus Eligible
• Relocation Expense Covered No
• Employee Type FT Exempt

YOUR ROLE

As an Agentic AI / LLM Engineer on the Coverage Intelligence team, you design and build the AI systems that turn complex insurance documents into reliable, explainable coverage intelligence inside the Digital Exchange, powered by DeX ai. You own problems end to end, from understanding unstructured documents to building agents that reason about coverage, and you hold that work to a measurable standard of accuracy. You work alongside our coverage and ontology specialists and help shape how the team builds with AI as the technology evolves.

WHAT YOU'LL DO

• Design, build and own production AI systems for document understanding and coverage analysis.
• Build agentic workflows that combine models, tools and human review, and recognize when a decision needs an expert.
• Ground outputs in source documents and domain knowledge so every result is traceable and explainable.
• Drive accuracy through evaluation, experimentation and error analysis against expert-labeled data.
• Make sound trade-offs across models and providers on accuracy, cost and latency.
• Partner with domain experts and engineers to turn expert judgment into durable system behavior.
• Help set the team's technical direction as LLM and agent capabilities evolve.
• Other duties and responsibilities as assigned.

Requirements

YOUR LOCATION

This is a fully remote role, with the exception of onboarding and optional in-office events.

SKILLS AND QUALIFICATIONS

• 5+ years' experience in software engineering or applied machine learning, including 2+ years building LLM applications in production.
• Strong Python skills, including typed schemas for structured output (e.g., Pydantic).
• Hands-on experience extracting structured data from complex documents using PDF parsing, OCR and vision-language models.
• Experience with agentic patterns such as tool use, multi-step workflows, self-verification and human-in-the-loop review.
• Experience with retrieval-augmented generation, embeddings and source-grounded output.
• A rigorous approach to evaluation: defining metrics, building test sets and doing systematic error analysis.
• Excellent written and verbal communication skills, including working directly with domain experts.
• Works autonomously with minimal direction, sets a clear path through ambiguity, and holds a high bar for quality.
• Thrives on the energy of a start up: comfortable with pace, changing priorities, and building the function while running it.
• Experience with insurance, legal, financial or other contract-heavy documents (strongly preferred).
• Experience with LangGraph or similar agent frameworks (preferred).
• Experience with LLM observability and evaluation tooling (preferred).

• Must be able to sit/stand/walk for prolonged periods of time (up to 8 hours per day), at a desk working on a computer.
• Must be able to use standard office equipment for extended periods of time, including but not limited to, a mouse, keyboard, phone and video conferencing.
• Must be able to travel to the Bold Penguin office in Dublin, Ohio for New Hire Onboarding & Orientation (2 days, 2 nights); travel costs are covered by Bold Penguin.
• Must be able to travel occasionally (up to 10%) for corporate events and team planning sessions.

Bold Penguin is the premier digital broker for commercial insurance. Their technology expands top-of-funnel demand and increases yield for enterprise partners. Through digital distribution, wholesale capabilities, and AI-driven insights, they transform the lifecycle of both simple and complex risks, from prospecting to placement.

We offer competitive compensation and progressive benefits that include:

• Medical, Dental, and Vision
• Flexible PTO Policy
• 401(k) with a company match
• Employee Assistance Program
• Disability and Life Benefits

Stay connected to the Glacier.

We have great SLACK channels for work and play. We also like to video conference and hold all-hands \"Waddles\" regularly.

Penguin bling.

Like swag themed after a certain Antarctic bird? Just. You. Wait.

Bold Penguin believes in inclusion. That's why we're proud to be an equal opportunity employer that considers all qualified applicants regardless of race, color, religion, gender identity or expression, sexual orientation, national origin, genetics, disability, age, or veteran status. To learn more about our results-focused culture and employee-focused perks, read more on our careers page .