Senior Applied AI Data Scientist – LLM/RAG for Healthcare

Definitive Healthcare Corp.; Woodbridge Township (NJ); On-site; USD 136,000 - 206,000; Full time

Benefits offered by this job

Healthcare benefits

Flexible paid time off

401(k) plan with employer contribution

Job summary

Definitive Healthcare Corp. is seeking a Senior Data Scientist to independently own the build and evaluation of AI solutions for an assigned use case, operating within the team's evaluation framework. The role applies professional judgement on technical approach, diagnosing issues and escalating decisions that affect strategy.

You will collaborate with engineering, product, and domain partners, maintain test assets, support production reliability, and communicate findings with clear tradeoffs.

Qualifications

• Bachelor's degree required; master's preferred.
• 5+ years of relevant experience in data science, machine learning, applied AI, software engineering, or related field.
• Hands-on experience building and evaluating LLM applications.

Responsibilities

• Own assigned workflows: Independently build, test, and improve LLM, RAG, and tool-using agent workflows for an assigned use case, using maintainable, tested Python code.
• Apply the evaluation framework: Select and apply evaluation criteria, baselines, and test datasets within the team's established evaluation approach; does not set that approach for the team.
• Select technical approaches: Compare models, embeddings, and retrieval methods for the assigned use case using evidence on quality, reliability, latency, and cost; document recommendations and tradeoffs for that project.
• Diagnose and resolve issues: Investigate retrieval misses, incorrect answers, and workflow failures within the assigned project; independently determine root cause and implement and validate the fix.
• Maintain and extend test assets: Build and update test cases, benchmark datasets, and regression checks for the assigned use case.
• Support production reliability: Monitor and investigate performance changes for workflows they own, and recommend and implement improvements within their project.
• Collaborate on delivery: Communicate findings and technical tradeoffs on the assigned project to immediate engineering, product, and domain partners.

Skills

Python programming

LLMs

Evaluation frameworks

Agent frameworks

Communication

Problem solving

Education

Bachelor's degree in quantitative/technical field

Master's degree preferred

Tools

Databricks

Delta Lake

Unity Catalog

Lakebase

Lakeflow Jobs

MLflow

Job description

Definitive Healthcare Corp. is seeking a Senior Data Scientist to independently own the build and evaluation of AI solutions for an assigned use case, operating within the team's evaluation framework. The role applies professional judgement on technical approach, diagnosing issues and escalating decisions that affect strategy.

You will collaborate with engineering, product, and domain partners, maintain test assets, support production reliability, and communicate findings with clear tradeoffs.