Senior AI Scientist, Applied AI: LLM/RAG for Healthcare

Definitive Healthcare, US; Framingham (MA); Hybrid; USD 136,000 - 206,000; Full time

Benefits offered by this job

Medical, dental, and vision insurance

Unlimited PTO

401(k) plan with employer match

Job summary

Definitive Healthcare in Framingham, MA is seeking a Senior Data Scientist to own the build and evaluation of AI solutions for an assigned use case, applying the team’s evaluation framework. You will design and test LLM workflows, embeddings, and RAG architectures using Python, with focus on quality, reliability, latency, and cost.

You will diagnose issues, compare models, and document tradeoffs while collaborating with engineering and product teams to continuously improve production systems.

Qualifications

• Bachelor's degree required in a quantitative or technical field.
• 5+ years of relevant experience in data science, ML, AI or related field.
• Hands-on experience building and evaluating LLM applications.

Responsibilities

• Own assigned workflows: build, test, and improve LLM/RAG/tool-using workflows in Python.
• Apply evaluation criteria, baselines, and test datasets within the team framework.
• Compare models, embeddings, and retrieval methods; document tradeoffs.
• Diagnose retrieval misses and workflow failures; implement fixes.
• Maintain test assets: test cases, benchmarks, regression checks.
• Support production reliability and monitor performance changes.
• Collaborate with engineering, product, and domain partners.

Skills

Python programming

LLM knowledge

Embeddings

Semantic search

RAG architectures

Test design

Communication

Education

Bachelor's degree required

Master's degree preferred

Tools

MLflow

Unity Catalog

Delta Lake

Lakebase

Lakeflow Jobs

Job description

Definitive Healthcare in Framingham, MA is seeking a Senior Data Scientist to own the build and evaluation of AI solutions for an assigned use case, applying the team’s evaluation framework. You will design and test LLM workflows, embeddings, and RAG architectures using Python, with focus on quality, reliability, latency, and cost.

You will diagnose issues, compare models, and document tradeoffs while collaborating with engineering and product teams to continuously improve production systems.