Data Scientist, Applied AI

definitivehc; United States; On-site; USD 111,000 - 167,000; Full time

Benefits offered by this job

Medical coverage

Dental coverage

Vision coverage

Unlimited PTO

401(k) with employer contribution

Job summary

Definitive Healthcare is seeking a Data Scientist to develop and test AI workflows, including LLMs and RAG, to support practical AI applications for healthcare data.

You will compare models, tune pipelines, and maintain evaluation assets while collaborating with engineering and product teams to deliver tangible AI solutions. A strong analytics background and hands-on AI experience are essential.

Qualifications

• Three+ years of experience in data science, ML, or applied AI.
• Hands-on experience developing applications with LLMs and evaluating outputs.
• Bachelor's degree in a quantitative or technical field.

Responsibilities

• Develop AI workflows: build and test LLM, RAG, and tool-using agent workflows for assigned use cases.
• Evaluate approaches: compare LLMs, embedding models, and retrieval methods for quality, reliability, latency, and cost.
• Maintain evaluation assets: create and update test cases, benchmark datasets, and regression checks; report results.
• Improve retrieval quality: develop and tune RAG pipelines including embeddings, retrieval, reranking, and grounding.
• Investigate failures: review incorrect answers, retrieval misses, and workflow failures; measure improvements.
• Support production workflows: use experimentation, tracing, and observability tools to monitor systems.
• Collaborate on delivery: work with engineering, product, and domain partners to clarify requirements and deliver AI solutions.

Skills

Python

Data analysis

Problem solving

Embeddings

Semantic search

Retrieval

RAG

Evaluation metrics

Education

Bachelor's degree in a quantitative field

Tools

PydanticAI

LangGraph

Job description

About Definitive Healthcare :

At Definitive Healthcare (NASDAQ: DH), we're passionate about turning data, analytics, and expertise into meaningful intelligence that helps our customers achieve success and shape the future of healthcare. We empower them to uncover the right markets, opportunities, and people-paving the way for smarter decisions and greater impact.

Headquartered just outside of Boston, Massachusetts, Definitive Healthcare operates across North America, Europe, and India, supporting a growing global client base of more than 2,400 customers since our founding in 2011.

We're also a great place to work. In 2024 and 2025, we earned multiple workplace honors, including Built In's 100 Best Places to Work in Boston (both years), a Stevie Bronze Award for Great Employers, and recognition as a Great Place to Work in India.

We foster a collaborative, inclusive culture where diverse perspectives drive innovation. Through programs like DefinitiveCares and our employee-led affinity groups we strive to promote connection, education, and inclusion.

About the position:

This role supports the team's work on practical generative AI applications. The Data Scientist implements and tests LLM, RAG, and agent-based workflows for assigned use cases under the direction of senior team members, producing evidence that informs team decisions about which approaches to use.

What y ou'll do :

Develop AI workflows: Build and test LLM, RAG, and tool-using agent workflows for assigned use cases.

Evaluate approaches: Compare LLMs, embedding models, and retrieval methods against quality, reliability, latency, and cost, using evaluation criteria and methods set by senior team members.

Maintain evaluation assets: Create and update representative test cases, benchmark datasets, and regression checks; report results and limitations.

Improve retrieval quality: Develop and tune RAG pipelines, including embeddings, retrieval, reranking, and grounding, based on evaluation results.

Investigate failures: Review incorrect answers, retrieval misses, and workflow failures; implement and measure improvements.

Support production workflows: Use experimentation, tracing, and observability tools to help monitor systems and investigate performance changes.

Collaborate on delivery: Work with engineering, product, and domain partners to clarify requirements, communicate findings, and deliver practical AI solutions.

What you'll need:

Education and experience

Bachelor's degree in a quantitative or technical field.

3+ years of experience in data science, machine learning, applied AI, or a related field.

Hands-on experience developing applications with LLMs and evaluating their outputs.

Technical and professional skills

Strong Python, data analysis, and problem-solving skills.

Understanding of embeddings, semantic search, retrieval, RAG and evaluation metrics.

Experience with an agent framework such as PydanticAI, LangGraph, or a similar tool.

Ability to design useful test cases, interpret evaluation results, and identify limitations.

Familiarity with cloud data or ML platforms, Git, testing, and collaborative software development.

Clear communication with technical and nontechnical colleagues.

Experience with Databricks, evaluation platforms, production deployment, fine-tuning, distributed processing, or healthcare data is beneficial but not required.

Autonomy, decisions, and scope

This is an individual contributor role with no people-management responsibilities.

Works independently on defined analyses and implementation tasks, with guidance on project priorities and broader technical direction.

Makes day-to-day decisions about experiments, test cases, and implementation details within agreed project goals.

Recommends model and retrieval improvements based on results; escalates major architecture changes, sensitive-data concerns, significant cost implications, and production release decisions to senior team members or the appropriate owners.

Influences assigned projects through implementation and evidence, rather than owning the evaluation strategy or technical direction across multiple projects.

Compensation and Benefits:

The salary range for this position is $111,000- $167,000 per year, which represents the base pay the company reasonably and in good faith expects to pay for this role. Actual pay within this range will be determined based on factors such as relevant experience, skills, and qualifications.

Depending on the position, employees may also be eligible to participate in a company bonus or commission plan. All employees are eligible for a comprehensive benefits package, including medical, dental, and vision coverage, unlimited paid time off, and participation in the company's 401(k) plan with employer contribution.

Why we love Definitive, and why you will too!

Industry leading products

Work hard, and have fun doing it

Incredibly fast growth means limitless opportunity

Flexible and dynamic culture

Wor