Graduate Intern - LLM Reliability and Uncertainty for AI Science Assistants

National Renewable Energy Laboratory; Colorado; On-site; USD 44,500 - 71,200; Part time

Benefits offered by this job

Medical Insurance

403(b) Employer Match

Sick leave

Annual salary range

Job summary

The National Renewable Energy Laboratory is offering a graduate internship focused on quantifying uncertainty and reducing hallucinations in LLM-based science assistants. You will explore multi-turn, agentic workflows, assess internal representations, and develop detectors and probing methods, with experiments on HPC systems.

You will contribute to documentation, reports, and presentations while building robust research code and evaluation pipelines under a multidisciplinary team in Golden,

Qualifications

• Minimum 3.0 cumulative GPA.
• Undergraduate: Full‑time student in a bachelor’s degree program; or earned a bachelor’s degree within the past 12 months.
• Graduate: Full‑time student in a master’s degree program; or earned a master’s degree within the past 12 months.
• Graduate + PhD: Completed master’s degree and enrolled as a PhD student.
• Eligible for an internship period of up to one year.
• Applicants must submit official or unofficial transcripts and a letter of recommendation (if selected).

Responsibilities

• Research and evaluate uncertainty quantification and hallucination detection methods for multi‑turn, agentic scientific workflows.
• Develop probing methods that predict, from a model’s internal representations, when a scientific task specification is incomplete or inconsistent and a clarifying question is warranted.
• Build and instrument evaluation pipelines that capture and analyse model internal states over multi‑turn scientific dialogue on HPC systems.
• Conduct experiments and analyse model behaviour across computational science domains and established benchmarks.
• Contribute to technical documentation, research reports, publications, and presentations summarising project progress and findings.
• Develop, test, and maintain high‑quality research code and evaluation pipelines.

Skills

LLMs familiarity

ML models

Uncertainty concepts

Open-weight LLMs

Python on HPC

Software engineering

Independent work

Education

Bachelor’s degree or equivalent

Master’s degree in a related field

PhD student enrollment

Tools

Python

HPC systems

GPU clusters

Deep learning frameworks

Job description

Job Title

Graduate Intern – LLM Reliability and Uncertainty for AI Science Assistants

Location

Golden, Colorado, USA

Position Type

Intern (Fixed Term)

Hours Per Week

40

Overview

The National Laboratory of the Rockies (NLR) hosts the AI, Learning and Intelligent Systems group, where cutting‑edge research on large language models for scientific applications is conducted. This internship invites a graduate researcher to develop methods for quantifying and mitigating uncertainty in LLM‑based science assistants.

Job Description

The intern will investigate uncertainty quantification and hallucination detection in multi‑turn, agentic scientific workflows, develop probing methods based on internal representations, build evaluation pipelines on HPC systems, conduct experiments across computational science domains, and disseminate findings through documentation, reports, and presentations.

Responsibilities

• Research and evaluate uncertainty quantification and hallucination detection methods for multi‑turn, agentic scientific workflows.
• Develop probing methods that predict, from a model’s internal representations, when a scientific task specification is incomplete or inconsistent and a clarifying question is warranted.
• Build and instrument evaluation pipelines that capture and analyse model internal states over multi‑turn scientific dialogue on HPC systems.
• Conduct experiments and analyse model behaviour across computational science domains and established benchmarks.
• Contribute to technical documentation, research reports, publications, and presentations summarising project progress and findings.
• Develop, test, and maintain high‑quality research code and evaluation pipelines.

Basic Qualifications

• Minimum 3.0 cumulative GPA.
• Undergraduate: Full‑time student in a bachelor’s degree program; or earned a bachelor’s degree within the past 12 months.
• Graduate: Full‑time student in a master’s degree program; or earned a master’s degree within the past 12 months.
• Graduate + PhD: Completed master’s degree and enrolled as a PhD student.
• Eligible for an internship period of up to one year.
• Applicants must submit official or unofficial transcripts and a letter of recommendation (if selected).

Additional Required Qualifications

• Familiarity with large language models, including agentic, tool‑using, or multi‑turn conversational LLM systems.
• Experience developing or evaluating machine learning models for classification, uncertainty estimation, or related tasks.
• Knowledge of probabilistic machine learning or uncertainty quantification concepts.
• Hands‑on experience with open‑weight LLMs and modern deep learning frameworks.
• Experience running Python code on HPC or multi‑GPU systems.
• Strong software engineering and debugging skills.
• Ability to work independently while collaborating effectively in a multidisciplinary research environment.

Preferred Qualifications

• Research experience related to hallucination detection, uncertainty quantification, interpretability, explainability, or trustworthy AI.
• Familiarity with representation probing or mechanistic interpretability methods.
• Experience with LLM benchmarking and evaluation, including multi‑turn or conversational agent evaluation and LLM‑as‑a‑judge protocols.
• Experience with scientific question‑answering systems, AI for science applications, or scientific agent frameworks.
• Coursework or research background in a computational science domain (e.g., fluid mechanics, solid mechanics, materials science, or numerical methods for PDEs).

Benefits

• Medical, dental, and vision insurance.
• 403(b) Employee Savings Plan with employer match.
• Sick leave (where required by law).
• Annual salary range: $44,500 – $71,200.

Equal Opportunity Employer

All qualified applicants will receive consideration for employment without regard to age (40 and over), color, disability, gender identity, genetic information, marital status, domestic partner status, military or veteran status, national origin/ancestry, race, religion, creed, sex (including pregnancy, childbirth, breastfeeding), sexual orientation, and any other applicable status protected by federal, state, or local laws.

Legal & Safety

Internships extending beyond six months require a federal Personal Identity Verification (PIV) card and a favorable background investigation. All employees must pass a pre‑employment drug test as required by federal law.