Applied AI Data Scientist – Generative AI & RAG Expert

Definitive Healthcare, US; Framingham (MA); Hybrid; USD 111,000 - 167,000; Full time

Benefits offered by this job

401(k) match

Healthcare benefits

Unlimited PTO

Job summary

Definitive Healthcare in Framingham, MA is seeking a Data Scientist to advance practical generative AI applications. You will implement and test LLM, RAG, and agent-based workflows under supervision, delivering evidence to guide decisions.

The role emphasizes building evaluation assets, tuning retrieval systems, and collaborating across engineering, product, and domain partners to deploy useful AI solutions in healthcare data contexts.

Qualifications

• Bachelor’s degree in a quantitative or technical field.
• 3+ years of experience in data science, machine learning, applied AI, or a related field.
• Hands-on experience developing applications with LLMs.

Responsibilities

• Develop AI workflows: Build and test LLM, RAG, and tool-using agent workflows for assigned use cases.
• Evaluate approaches: Compare LLMs, embedding models, and retrieval methods against quality, reliability, latency, and cost, using evaluation criteria and methods set by senior team members.
• Maintain evaluation assets: Create and update representative test cases, benchmark datasets, and regression checks; report results and limitations.
• Improve retrieval quality: Develop and tune RAG pipelines, including embeddings, retrieval, reranking, and grounding, based on evaluation results.
• Investigate failures: Review incorrect answers, retrieval misses, and workflow failures; implement and measure improvements.
• Support production workflows: Use experimentation, tracing, and observability tools to help monitor systems and investigate performance changes.
• Collaborate on delivery: Work with engineering, product, and domain partners to clarify requirements, communicate findings, and deliver practical AI solutions.

Skills

Python

Data analysis

Problem solving

LLM development

Communication

Education

Bachelor's degree in a quantitative or technical field

Tools

PydanticAI

LangGraph

Databricks

Job description

Definitive Healthcare in Framingham, MA is seeking a Data Scientist to advance practical generative AI applications. You will implement and test LLM, RAG, and agent-based workflows under supervision, delivering evidence to guide decisions.

The role emphasizes building evaluation assets, tuning retrieval systems, and collaborating across engineering, product, and domain partners to deploy useful AI solutions in healthcare data contexts.