AI Engineer (LLMs for Healthcare)

keeblerhealth; Durham (NC); On-site; USD 140,000 - 190,000; Full time

Benefits offered by this job

Competitive salary and benefits

Fast-paced innovative environment

Professional growth opportunities

Collaborative team culture

Meaningful impact on healthcare

Job summary

Keebler Health is seeking a mid to senior AI Engineer to fine-tune LLMs, develop prompt strategies, and deploy RAG-based healthcare solutions. You will collaborate with clinicians and engineers to translate domain knowledge into scalable AI workflows, with emphasis on HIPAA-compliant data handling.

You will build production-grade ML pipelines, monitor models in deployment, and drive continuous improvement using modern MLOps practices and cloud infrastructure.

Qualifications

• Proven experience in LLM fine-tuning and advanced prompt engineering.
• Strong Python and ML framework experience (HuggingFace, PyTorch).
• Familiarity with healthcare workflows and regulatory requirements (HIPAA, FHIR).
• Hands-on experience with Retrieval-Augmented Generation (RAG).
• Ability to evaluate AI models using precision, recall, and other metrics.

Responsibilities

• Fine-tune and optimize large language models for healthcare applications.
• Develop and apply advanced prompt engineering techniques for clinical scenarios.
• Implement RAG systems to improve knowledge retrieval from large datasets.
• Work with knowledge graphs to organize healthcare data for decision-making.
• Evaluate models using metrics to ensure robustness and reliability.

Skills

LLM fine-tuning

Prompt engineering

Python

HuggingFace

PyTorch

RAG techniques

HIPAA/FHIR familiarity

Performance metrics (precision/recall)

Tools

Hugging Face transformers

PyTorch

Docker

Kubernetes

MLflow

Langfuse

Job description

About Keebler Health

Keebler Health is building the operating system for value-based care. Our mission is to help risk-bearing healthcare organizations thrive in value-based arrangements by unlocking the full power of their data. We empower leading primary care groups, ACOs, and health plans to act on real-time insights that improve outcomes, reduce costs, and fuel sustainable growth.

We're a fast-moving, high-performing team, and we’re looking for people who share our bias toward speed, urgency, and excellence. As a member of the team, you won’t just write code or stay in your lane—you’ll shape critical systems, innovate quickly, and set a high bar for a product that supports the future of U.S. healthcare.

About the role

We are seeking a talented and motivated mid to senior level AI Engineer with expertise in developing and fine-tuning large language models (LLMs), healthcare workflows, and AI/ML engineering best practices. The ideal candidate will bring a deep understanding of healthcare-specific challenges and modern AI techniques to drive innovation in Value-Based Care solutions. Level and salary will commensurate with experience.

Key Responsibilities

AI/ML Engineering

• Fine-tune and optimize large language models (LLMs) to address specific healthcare applications.
• Develop and apply advanced prompt engineering techniques to enhance model outputs for clinical scenarios.
• Implement Retrieval-Augmented Generation (RAG) systems to improve knowledge retrieval from large datasets.
• Work with knowledge graphs to organize and integrate healthcare-specific data for enhanced decision-making.
• Evaluate black-box models using precision, recall, and other performance metrics, ensuring robustness and reliability.

Healthcare Expertise

• Collaborate with healthcare professionals to understand workflows and identify opportunities for AI-driven enhancements.
• Design and build AI models that align with healthcare standards and regulations (e.g., HIPAA compliance).
• Integrate domain-specific knowledge of healthcare data, including FHIR and interoperability standards, into AI solutions.

MLOps & Deployment

• Develop and maintain scalable, production-ready AI pipelines using MLOps tools.
• Deploy and monitor AI models in production environments to ensure performance and compliance.
• Optimize infrastructure for efficient training, testing, and deployment of models.

Innovation and Optimization

• Stay at the forefront of advancements in AI, especially in healthcare applications.
• Identify and resolve performance bottlenecks in AI workflows.
• Explore emerging trends and technologies in LLMs and healthcare to continually improve solutions.

Collaboration and Impact

• Partner with cross-functional teams, including data engineers and clinicians, to ensure seamless integration of AI into healthcare workflows.
• Communicate technical results and insights effectively to non-technical stakeholders.

Required Qualifications

• Proven experience in LLM fine-tuning and advanced prompt engineering.
• Strong background in Python and modern ML frameworks (e.g., Huggingface, pyTorch).
• Familiarity with healthcare workflows and regulatory requirements (e.g., HIPAA, FHIR standards).
• Hands-on experience with retrieval-augmented generation (RAG) techniques.
• Expertise in evaluating AI models using performance metrics like precision, and recall.

Preferred Skills

• Experience with MLOps frameworks such as MLflow, Langfuse, or similar tools.
• Understanding of healthcare data standards, including HL7 and HEDIS metrics.
• Strong problem-solving skills in integrating AI with complex healthcare datasets.
• Familiarity with cloud platforms (e.g., AWS, GCP, or Azure) and containerization (Docker, Kubernetes).

When applying

In addition to your resume, also include:

• A highly personalized, bold, and hilarious “Keebler Health–style” introduction that grabs attention - outgoing, fun, and uniquely you (not uniquely ChatGPT). Think: confident, high-energy, slightly irreverent (but still professional), with a smart nod to healthcare, value-based care, and the fact that we’re building something real.

What We Offer

• Competitive salary and benefits package.
• Opportunity to work in a fast-paced, innovative environment.
• Professional growth and development opportunities.
• Collaborative and supportive team culture.
• Chance to make a meaningful impact on the healthcare industry.