Senior AI Engineer LLM and Agentic Workflows

Ellkay, Llc; Indiana; Sur place; USD 140 000 - 190 000; Plein temps

Avantages offerts par ce poste

Office locations

Smoke‑Free Workplace

Equal opportunity employer

Résumé du poste

ELLKAY is seeking a Senior AI Engineer to build production LLM applications and agent workflows for implementation assistance, connector discovery, migration operations, and archive access in Indiana. You will implement retrieval, tool integration, workflow state, citations, and automated evaluation while connecting assistants to existing ELLKAY services with strict data boundaries.

Ideal candidates have 5+ years in software/AI engineering, strong Python, REST APIs, asynchronous programming, and

Qualifications

• 5+ years in software/AI engineering, including production LLM applications and backend services.
• Strong Python, REST APIs, asynchronous programming, data handling, software testing, and production debugging.
• Hands‑on RAG, hybrid/vector search, embeddings, reranking, structured outputs, context management, and tool calling.
• Experience implementing agent state, timeouts, bounded retries, approval checkpoints, fallback, and duplicate‑action prevention.
• Ability to evaluate retrieval relevance, groundedness, tool selection, task completion, latency, and unit cost using representative tests.
• Experience deploying services to a cloud environment and implementing authorization‑aware retrieval and PHI‑safe telemetry.

Responsabilités

• Build RAG and tool-calling workflows for implementation assistance, connector capability discovery, error explanation, and guided migration operations.
• Create context assembly from approved documentation, run status, mapping metadata, and evidence. Return citations and structured outputs.
• Implement MCP clients/tool integrations and thin adapters over platform contracts. Handle async request IDs, retries, timeouts, and duplicate action prevention with the orchestration team.
• Implement permission‑aware retrieval, approval checkpoints, prompt‑injection defenses, bounded tool use, and explicit failure or abstention behavior.
• Capture agent traces and user feedback, and add regression tests for prompts, tool selection, retrieval quality, and task completion.
• Improve retrieval and context assembly using measured failures. Evaluate rerankers or model adaptation when simpler prompting and retrieval changes do not meet quality needs.
• Partner with Data Platform and product engineers, clinical/data SMEs, DME/QA, and security teams. Build reusable services for ELLKAY products and preserve source evidence, approved data‑use rights, and tenant isolation.

Connaissances

5+ years in software/AI engineering

Python

REST APIs

asynchronous programming

data handling

software testing

production debugging

RAG

vector search

embeddings

tool calling

agent state management

retrieval evaluation

cloud deployment

PHI-safe telemetry

Formation

Relevant degree or equivalent practical experience

Outils

FastAPI

AWS

Docker/Kubernetes

HL7/FHIR

Asana/JTS

Description du poste

If you are unable to complete this application due to a disability, contact this employer to ask for an accommodation or an alternative application process.

Senior AI Engineer LLM and Agentic Workflows

IN

3 days ago Requisition ID: 1398

ELLKAY started out providing connectivity solutions to laboratories and within a few years, grew to also provide data management solutions to ambulatory organizations. ELLKAY is now a trusted data management partner in five healthcare segments. ELLKAY’s solutions continue to serve laboratories and ambulatory practices and have expanded to empower hospitals and health systems, healthcare IT vendors, ambulatory practices, health plans, and other healthcare organizations with cutting-edge technologies and solutions that drive their growth and interoperability strategies. Today, ELLKAY remains true to our core values, building strong partner relationships and offering unparalleled service and support while providing innovative, scalable solutions to the challenges our customers face in today’s data-rich world. ELLKAY's experience, customer-focused approach, and reputation for innovation, speed, and accuracy differentiate ELLKAY as a premier partner for your interoperability needs and data management strategy. Job Description: Build production LLM applications and agent workflows for implementation assistance, connector capability discovery, migration operations, and archive access. The engineer will implement retrieval, tool integration, workflow state, citations, approval checkpoints, and automated evaluation. The role connects assistants to existing ELLKAY services and the Central DME Portal while preserving tenant, purpose, and data-access boundaries.

Essential Duties and Responsibilities:

• Build RAG and tool-calling workflows for implementation assistance, connector capability discovery, error explanation, and guided migration operations.
• Create context assembly from approved documentation, run status, mapping metadata, and evidence. Return citations and structured outputs.
• Implement MCP clients/tool integrations and thin adapters over platform contracts. Handle async request IDs, retries, timeouts, and duplicate action prevention with the orchestration team.
• Implement permission‑aware retrieval, approval checkpoints, prompt‑injection defenses, bounded tool use, and explicit failure or abstention behavior.
• Capture agent traces and user feedback, and add regression tests for prompts, tool selection, retrieval quality, and task completion.
• Improve retrieval and context assembly using measured failures. Evaluate rerankers or model adaptation when simpler prompting and retrieval changes do not meet quality needs.
• Partner with Data Platform and product engineers, clinical/data SMEs, DME/QA, and security teams. Build reusable services for ELLKAY products and preserve source evidence, approved data‑use rights, and tenant isolation. Model or agent proposals follow established human approval and platform release workflows.

Qualifications

• 5+ years in software/AI engineering, including production LLM applications and backend services. The total experience requirement does not imply five years in generative AI alone.
• Strong Python, REST APIs, asynchronous programming, data handling, software testing, and production debugging.
• Hands‑on RAG, hybrid/vector search, embeddings, reranking, structured outputs, context management, and tool calling.
• Experience implementing agent state, timeouts, bounded retries, approval checkpoints, fallback, and duplicate‑action prevention.
• Ability to evaluate retrieval relevance, groundedness, tool selection, task completion, latency, and unit cost using representative tests.
• Experience deploying services to a cloud environment and implementing authorization‑aware retrieval and PHI‑safe telemetry. A relevant degree or equivalent practical experience is acceptable.

Preferred qualifications

MCP, LangGraph or equivalent agent orchestration, FastAPI, AWS, Docker/Kubernetes, HL7/FHIR familiarity, enterprise authorization patterns, and integrations with project or engineering systems such as Asana/JTS.

Expected outcomes

• First 90 days: deliver a synthetic‑data agent prototype with tool contracts, traces, and regression tests covering correct execution and failure cases.
• By six months: pilot no‑PHI capability, status, and failure‑explanation tools with a design partner, recording task completion, groundedness, response time, and cost per task.
• By 12 months: target at least 80% successful completion of agreed supported tasks on a held‑out benchmark within approved cost/latency limits. Require passing authorization tests, traceability for all tool actions, and confirmation for exposed action/bulk tools.
• Quality, performance, and business‑impact targets will be calibrated to the selected use cases and their initial baselines. Delivery depends on the corresponding platform capabilities and approved customer pilot scope.

Engineering approach

Use rules, classical ML, retrieval, or LLMs according to measured task performance. Document model/data limitations and route uncertain outputs for review. Retain versioned evaluations and release evidence, and improve the service through production feedback. Training on customer data requires approved rights. Clinical delivery paths remain independent of AI processing.
Additional Information:

ELLKAY is committed to fostering a collaborative and high‑performance work environment that supports innovation, teamwork, and professional growth. Most roles are designed to operate from our office locations to encourage effective collaboration and engagement across teams.
Any alternative work arrangements may be considered at the company’s discretion based on role requirements and business needs.
For more information about our company, please visit www.ELLKAY.com .
ELLKAY is a Smoke‑Free Workplace.

ELLKAY, LLC provides equal employment opportunities to all employees and applicants for employment and prohibits discrimination and harassment of any type without regard to race, color, religion, age, sex, national origin, disability status, genetics, protected veteran status, sexual orientation, gender identity or expression, or any other characteristic protected by federal, state or local laws.