AI Quality & Evaluation Scientist for Clinical LLMs

Bioscope.ai, Inc.; Boston (MA); Vor Ort; USD 100.000 - 130.000; Vollzeit

Zusammenfassung

A leading clinical AI company is seeking a Data Scientist responsible for ensuring the quality and reliability of AI outputs. You will design automated evaluation pipelines, develop uncertainty quantification systems, and build comprehensive frameworks that combine automated assessments with clinician validations. The ideal candidate has expertise in deep learning frameworks such as PyTorch, experience with model evaluation metrics, and excellent communication skills. A Master's degree in a related field is preferred.

Qualifikationen

• Strong foundation in deep learning frameworks (PyTorch) and LLM architectures.
• Experience with model evaluation, benchmarking, and quality metrics.
• Proficiency in Python and modern ML development tools.

Aufgaben

• Design and implement automated evaluation pipelines to assess AI output quality.
• Develop uncertainty quantification systems correlating confidence scores with accuracy.
• Build evaluation frameworks combining automated assessment and clinician-validated cases.

Kenntnisse

Deep learning frameworks (PyTorch)

Model evaluation and quality metrics

Proficiency in Python

Statistical analysis

Communication skills

Ausbildung

Master's degree in Computer Science, Machine Learning, Statistics, or related field

Jobbeschreibung

A leading clinical AI company is seeking a Data Scientist responsible for ensuring the quality and reliability of AI outputs. You will design automated evaluation pipelines, develop uncertainty quantification systems, and build comprehensive frameworks that combine automated assessments with clinician validations. The ideal candidate has expertise in deep learning frameworks such as PyTorch, experience with model evaluation metrics, and excellent communication skills. A Master's degree in a related field is preferred.