CDS Faculty Member Tim G. J. Rudner Receives Major Grant to Study Uncertainty Quantification in Large Language Models

Large language models (LLMs) often express high confidence even when providing incorrect answers. This fundamental challenge in AI reliability motivated CDS faculty member Tim G. J. Rudner to develop new approaches for helping AI systems estimate their own uncertainty. The Georgetown University Center for Security and Emerging Technology (CSET) recently awarded Rudner a $694,713 Foundational Research Grant to pursue this work.

Rudner’s two-year project aims to establish frameworks for uncertainty quantification in large language models, particularly in open-ended settings. “If the model is able to reliably say ‘Hey, this is my response, but I have a high degree of uncertainty about it,’ then users can take that into account,” Rudner said.

“This is a very exciting and promising area of research,” Rudner added, “but it’s still very much in its infancy.”

Initial results from this research have been promising. In a paper, “SCIURus: Shared Circuits for Interpretable Uncertainty Representations in Language Models,” Rudner, NYU master’s student Carter Teplica, and collaborators from Yale investigated internal mechanisms of LLMs responsible for uncertainty expressions and found evidence that, for certain tasks, language models use shared neural circuits for both generating answers and expressing their confidence in those answers, rather than having separate mechanisms for each. The paper will be presented at the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics (NAACL 2025).

The grant, which supported this research, and is part of CSET’s program to explore AI’s potential national security implications, represents an unusual achievement for an early-career researcher. The funding will support both computational resources for working with frontier AI models and personnel to assist with Rudner’s research.

Rudner’s previous work on uncertainty-aware priors for neural networks established standards for uncertainty quantification in computer vision and language classification tasks. However, he notes that generative models like ChatGPT pose unique challenges, as they can express uncertainty in multiple ways that often proved unreliable.

“It’s an open question how we can best quantify a generative language model’s uncertainty about its responses to user prompts,” Rudner said. “For example, should we enable language model’s to accurately verbalize their uncertainty or should we develop tools that extract uncertainty estimates from language models’ internal representations?”

The project builds on Rudner’s long-standing collaboration with CSET, where he has published several papers on AI policy and governance since 2019. His technical research has focused on developing methodological and empirical insight that can inform the design of standards and regulation around AI reliability, safety, transparency and fairness.

By Stephen Thomas