Senior AI Engineer: LLM & Agentic Workflows

Ellkay, Llc; Indiana; On-site; USD 140,000 - 190,000; Full time

Benefits offered by this job

Office locations

Smoke‑Free Workplace

Equal opportunity employer

Job summary

ELLKAY is seeking a Senior AI Engineer to build production LLM applications and agent workflows for implementation assistance, connector discovery, migration operations, and archive access in Indiana. You will implement retrieval, tool integration, workflow state, citations, and automated evaluation while connecting assistants to existing ELLKAY services with strict data boundaries.

Ideal candidates have 5+ years in software/AI engineering, strong Python, REST APIs, asynchronous programming, and

Qualifications

• 5+ years in software/AI engineering, including production LLM applications and backend services.
• Strong Python, REST APIs, asynchronous programming, data handling, software testing, and production debugging.
• Hands‑on RAG, hybrid/vector search, embeddings, reranking, structured outputs, context management, and tool calling.
• Experience implementing agent state, timeouts, bounded retries, approval checkpoints, fallback, and duplicate‑action prevention.
• Ability to evaluate retrieval relevance, groundedness, tool selection, task completion, latency, and unit cost using representative tests.
• Experience deploying services to a cloud environment and implementing authorization‑aware retrieval and PHI‑safe telemetry.

Responsibilities

• Build RAG and tool-calling workflows for implementation assistance, connector capability discovery, error explanation, and guided migration operations.
• Create context assembly from approved documentation, run status, mapping metadata, and evidence. Return citations and structured outputs.
• Implement MCP clients/tool integrations and thin adapters over platform contracts. Handle async request IDs, retries, timeouts, and duplicate action prevention with the orchestration team.
• Implement permission‑aware retrieval, approval checkpoints, prompt‑injection defenses, bounded tool use, and explicit failure or abstention behavior.
• Capture agent traces and user feedback, and add regression tests for prompts, tool selection, retrieval quality, and task completion.
• Improve retrieval and context assembly using measured failures. Evaluate rerankers or model adaptation when simpler prompting and retrieval changes do not meet quality needs.
• Partner with Data Platform and product engineers, clinical/data SMEs, DME/QA, and security teams. Build reusable services for ELLKAY products and preserve source evidence, approved data‑use rights, and tenant isolation.

Skills

5+ years in software/AI engineering

Python

REST APIs

asynchronous programming

data handling

software testing

production debugging

RAG

vector search

embeddings

tool calling

agent state management

retrieval evaluation

cloud deployment

PHI-safe telemetry

Education

Relevant degree or equivalent practical experience

Tools

FastAPI

AWS

Docker/Kubernetes

HL7/FHIR

Asana/JTS

Job description

ELLKAY is seeking a Senior AI Engineer to build production LLM applications and agent workflows for implementation assistance, connector discovery, migration operations, and archive access in Indiana. You will implement retrieval, tool integration, workflow state, citations, and automated evaluation while connecting assistants to existing ELLKAY services with strict data boundaries.

Ideal candidates have 5+ years in software/AI engineering, strong Python, REST APIs, asynchronous programming, and