Applied AI Data Scientist — LLM, RAG & Agent Workflows

Definitive Healthcare Corp.; Woodbridge Township (NJ); On-site; USD 111,000 - 167,000; Full time

Benefits offered by this job

Medical, dental, and vision coverage

401(k) plan with employer contribution

Flexible paid time off

Job summary

Definitive Healthcare in the United States seeks a Data Scientist to advance practical generative AI applications. The role implements and tests LLM, RAG, and agent-based workflows under senior guidance, producing evidence to inform decisions about approach selection.

You'll evaluate models, tune retrieval pipelines, and support production workflows while collaborating with engineering, product, and domain partners to deliver impactful AI solutions in healthcare data.

Qualifications

• Bachelor’s degree in a quantitative or technical field.
• 3+ years of experience in data science, machine learning, applied AI, or a related field.
• Hands-on experience developing applications with LLMs and evaluating their outputs.

Responsibilities

• Develop AI workflows: Build and test LLM, RAG, and tool-using agent workflows for assigned use cases.
• Evaluate approaches: Compare LLMs, embedding models, and retrieval methods against quality, reliability, latency, and cost.
• Maintain evaluation assets: Create and update representative test cases, benchmark datasets, and regression checks; report results and limitations.
• Improve retrieval quality: Develop and tune RAG pipelines, including embeddings, retrieval, reranking, and grounding, based on evaluation results.
• Investigate failures: Review incorrect answers, retrieval misses, and workflow failures; implement and measure improvements.
• Support production workflows: Use experimentation, tracing, and observability tools to help monitor systems and investigate performance changes.
• Collaborate on delivery: Work with engineering, product, and domain partners to clarify requirements, communicate findings, and deliver practical AI solutions.

Skills

Python

Data analysis

Problem solving

Embeddings

Semantic search

Retrieval

RAG

Evaluation metrics

Agent frameworks (PydanticAI/LangGraph

Git

Databricks

Education

Bachelor's degree in a quantitative or technical field

Tools

PydanticAI

LangGraph

Databricks

Git

Job description

Definitive Healthcare in the United States seeks a Data Scientist to advance practical generative AI applications. The role implements and tests LLM, RAG, and agent-based workflows under senior guidance, producing evidence to inform decisions about approach selection.

You'll evaluate models, tune retrieval pipelines, and support production workflows while collaborating with engineering, product, and domain partners to deliver impactful AI solutions in healthcare data.