Applied AI Data Scientist: LLM/RAG, Unlimited PTO

definitivehc; United States; On-site; USD 111,000 - 167,000; Full time

Benefits offered by this job

Medical coverage

Dental coverage

Vision coverage

Unlimited PTO

401(k) with employer contribution

Job summary

Definitive Healthcare is seeking a Data Scientist to develop and test AI workflows, including LLMs and RAG, to support practical AI applications for healthcare data.

You will compare models, tune pipelines, and maintain evaluation assets while collaborating with engineering and product teams to deliver tangible AI solutions. A strong analytics background and hands-on AI experience are essential.

Qualifications

• Three+ years of experience in data science, ML, or applied AI.
• Hands-on experience developing applications with LLMs and evaluating outputs.
• Bachelor's degree in a quantitative or technical field.

Responsibilities

• Develop AI workflows: build and test LLM, RAG, and tool-using agent workflows for assigned use cases.
• Evaluate approaches: compare LLMs, embedding models, and retrieval methods for quality, reliability, latency, and cost.
• Maintain evaluation assets: create and update test cases, benchmark datasets, and regression checks; report results.
• Improve retrieval quality: develop and tune RAG pipelines including embeddings, retrieval, reranking, and grounding.
• Investigate failures: review incorrect answers, retrieval misses, and workflow failures; measure improvements.
• Support production workflows: use experimentation, tracing, and observability tools to monitor systems.
• Collaborate on delivery: work with engineering, product, and domain partners to clarify requirements and deliver AI solutions.

Skills

Python

Data analysis

Problem solving

Embeddings

Semantic search

Retrieval

RAG

Evaluation metrics

Education

Bachelor's degree in a quantitative field

Tools

PydanticAI

LangGraph

Job description

Definitive Healthcare is seeking a Data Scientist to develop and test AI workflows, including LLMs and RAG, to support practical AI applications for healthcare data.

You will compare models, tune pipelines, and maintain evaluation assets while collaborating with engineering and product teams to deliver tangible AI solutions. A strong analytics background and hands-on AI experience are essential.