Inpatient Nursing AI Evaluator and Data Annotator

Mercor; New York (NY); On-site; USD 60,000 - 90,000; Full time

Job summary

Obsidian is hiring experienced Adult Inpatient Nurses (RNs) to help train and evaluate AI systems used in clinical and healthcare settings. You will translate bedside documentation practices into structured feedback for AI and work on projects requiring clinical judgment and attention to detail.

You will annotate and structure inpatient nursing data for AI training datasets, review AI outputs for accuracy, and collaborate with technical teams to improve model performance and benchmarks.

Qualifications

• Active RN license (U.S., outside California).
• Recent adult acute care inpatient bedside experience, ideally in a non-procedural area of specialty.
• Experience within the last 5–10 years, reflecting familiarity with current workflows and documentation practices (current documentation experience is prioritized over total years of nursing experience).
• Comfortable performing and documenting comprehensive nursing assessments.
• Comfortable following detailed annotation guidelines consistently.
• Excellent written communication skills and responsiveness to feedback.
• Ability to work independently and meet deadlines.
• Epic EHR experience (preferred).
• Ability to navigate transcripts and use AI-assisted tools while verifying outputs.
• Experience reviewing charts for quality improvement or clinical informatics.

Responsibilities

• Review and evaluate AI-generated clinical outputs based on nursing flowsheet documentation
• Validate accuracy, completeness, and adherence to current documentation standards
• Annotate and structure inpatient nursing assessment data for AI training datasets
• Provide expert feedback on nursing assessments and documentation practices
• Identify gaps, inconsistencies, or risks in AI-generated responses
• Ask clarifying questions when annotation guidance is ambiguous and contribute to guideline refinement
• Collaborate with technical teams to improve model performance
• Contribute to the development of high-quality clinical benchmarks

Skills

RN license (US)

Adult acute care

Documentation familiarity

Annotation guidelines

Written communication

Independent work / deadlines

Job description

Obsidian is hiring experienced Adult Inpatient Nurses (RNs) to help train and evaluate AI systems used in clinical and healthcare settings. You will translate bedside documentation practices into structured feedback for AI and work on projects requiring clinical judgment and attention to detail.

You will annotate and structure inpatient nursing data for AI training datasets, review AI outputs for accuracy, and collaborate with technical teams to improve model performance and benchmarks.