Senior Data Scientist, Applied AI

definitivehc; United States; On-site; USD 136,000 - 206,000; Full time

Job summary

Definitive Healthcare, headquartered near Boston, seeks a Senior Data Scientist to own AI evaluation for healthcare-focused use cases. You will build and optimize LLM, RAG, and tool-using workflows in Python, leveraging Databricks to deliver reliable, scalable AI solutions.

You will evaluate models, embeddings, and retrieval methods, document tradeoffs, and collaborate with engineering and product teams to ensure high quality and cost-conscious deployments.

Qualifications

• Bachelor's degree required; master's degree in a quantitative or technical field preferred.
• 5+ years of relevant experience in data science, machine learning, applied AI, software engineering, or a related field.
• Hands-on experience building and evaluating LLM applications.
• Strong Python programming, software engineering and testing, and analytical problem-solving skills.
• Strong knowledge of LLMs, embeddings, semantic search, retrieval systems, and RAG architectures.
• Experience with Databricks, including Delta Lake, Unity Catalog, Lakebase, Lakeflow Jobs, and MLflow.
• Ability to design representative evaluations for an assigned use case, analyze errors, and use results to improve a system.
• Ability to weigh quality, reliability, latency, and cost when recommending an approach for their own project.
• Clear communication and effective collaboration with technical and nontechnical partners.
• Experience with agent frameworks and AI evaluation platforms.
• Experience with healthcare data is preferred, but not required.

Responsibilities

• Own assigned workflows: build, test, and improve LLM, RAG, and tool-using agent workflows with maintainable Python code.
• Apply the evaluation framework by selecting criteria, baselines, and test datasets.
• Select technical approaches by comparing models, embeddings, and retrieval methods with evidence on quality, latency, and cost.
• Diagnose and resolve issues: investigate retrieval misses, incorrect answers, and workflow failures, and implement fixes.
• Maintain and extend test assets: create test cases, benchmark datasets, and regression checks.
• Support production reliability: monitor performance and implement improvements.
• Collaborate with engineering, product, and domain partners to communicate findings and tradeoffs.

Skills

Python programming

Software engineering

Testing

Analytical problem-solving

LLMs

Embeddings

Semantic search

Retrieval systems

RAG architectures

Agent frameworks

AI evaluation platforms

Healthcare data

Education

Bachelor's degree

Master's degree

Tools

Databricks

Delta Lake

Unity Catalog

Lakebase

Lakeflow Jobs

MLflow

Job description

About Definitive Healthcare :

At Definitive Healthcare (NASDAQ: DH), we're passionate about turning data, analytics, and expertise into meaningful intelligence that helps our customers achieve success and shape the future of healthcare. We empower them to uncover the right markets, opportunities, and people-paving the way for smarter decisions and greater impact.

Headquartered just outside of Boston, Massachusetts, Definitive Healthcare operates across North America, Europe, and India, supporting a growing global client base of more than 2,400 customers since our founding in 2011.

We're also a great place to work. In 2024 and 2025, we earned multiple workplace honors, including Built In's 100 Best Places to Work in Boston (both years), a Stevie Bronze Award for Great Employers, and recognition as a Great Place to Work in India.

We foster a collaborative, inclusive culture where diverse perspectives drive innovation. Through programs like DefinitiveCares and our employee-led affinity groups we strive to promote connection, education, and inclusion.

About the position:

This role independently owns the build and evaluation of AI solutions for an assigned use case, operating within the team's established evaluation framework and practices. The Senior Data Scientist exercises professional judgment on technical approach, issue diagnosis, and implementation within their assigned project, escalating decisions that would affect the team's overall evaluation strategy, span multiple projects, or carry material risk. Its impact is measured by the quality and reliability of the systems and evidence it delivers for its assigned use case.

What y ou'll do:

• Own assigned workflows: Independently build, test, and improve LLM, RAG, and tool-using agent workflows for an assigned use case, using maintainable, tested Python code.
• Apply the evaluation framework: Select and apply evaluation criteria, baselines, and test datasets within the team's established evaluation approach; does not set that approach for the team.
• Select technical approaches: Compare models, embeddings, and retrieval methods for the assigned use case using evidence on quality, reliability, latency, and cost; document recommendations and tradeoffs for that project.
• Diagnose and resolve issues: Investigate retrieval misses, incorrect answers, and workflow failures within the assigned project; independently determine root cause and implement and validate the fix.
• Maintain and extend test assets: Build and update test cases, benchmark datasets, and regression checks for the assigned use case.
• Support production reliability: Monitor and investigate performance changes for workflows they own, and recommend and implement improvements within their project.
• Collaborate on delivery: Communicate findings and technical tradeoffs on the assigned project to immediate engineering, product, and domain partners.

What you'll need:

Education and experience

• Bachelor's degree required; master's degree in a quantitative or technical field preferred.
• 5+ years of relevant experience in data science, machine learning, applied AI, software engineering, or a related field.
• Hands-on experience building and evaluating LLM applications.

Technical and professional skills

• Strong Python programming, software engineering and testing, and analytical problem-solving skills.
• Strong knowledge of LLMs, embeddings, semantic search, retrieval systems, and RAG architectures.
• Strong hands-on experience with Databricks, including Delta Lake, Unity Catalog, Lakebase, Lakeflow Jobs, and MLflow, for building and deploying scalable, production-grade data and AI solutions.
• Ability to design representative evaluations for an assigned use case, analyze errors, and use results to improve a system.
• Ability to weigh quality, reliability, latency, and cost when recommending an approach for their own project.
• Clear communication and effective collaboration with technical and nontechnical partners.
• Experience with agent frameworks and AI evaluation platforms.
• Experience with healthcare data is preferred, but not required.

Autonomy, decisions, and scope

• This is an individual contributor role with no people-management responsibilities.
• Operates independently on an assigned project within the team's established evaluation framework and practices.
• Exercises professional judgment on technical approach, root-cause diagnosis, and issue resolution within their assigned project.
• Escalates decisions involving changes to the team's evaluation approach, significant business risk, sensitive data, material cost commitments, or production release approval to the Manager or appropriate owner.

Compensation and Benefits:

The salary range for this position is $136,000 - $206,000 per year, which represents the base pay the company reasonably and in good faith expects to pay for this role. Actual pay within this range will be determined based on factors such as relevant experience, skills, and qualific