CDS Assistant Professor Andrew Gordon Wilson Wins NSF CAREER Award
CDS Assistant Professor of Computer Science and Data Science, Andrew Gordon Wilson, recently received the National Science Foundation CAREER Award, under the Faculty Early Career Development Program (CAREER) Program, for his project “New Frontiers in Bayesian Deep Learning”.
Bayesian methods express the belief that we should represent all possible solutions to a given problem in making predictions, and combine them by their posterior probabilities, rather than bet everything on the most likely solution. These methods are particularly compelling in deep neural networks, because neural networks can represent many different solutions to a given problem, corresponding to different settings of their parameters.
The objective of “New Frontiers” is to develop new foundations in Bayesian deep learning that will guide the trajectory of the broader deep learning community at large. The research agenda argues we should embrace flexibility, since it reflects the honest belief that real-world processes are highly sophisticated, so long as we have reasonable inductive biases. From a Bayesian perspective, the inductive biases represent which solutions we believe are likely before we see the data.
The research consists of three strategic parts. The first part introduces new priors that naturally enable us to specify our inductive biases, encoding our high-level beliefs before we see the data. For example, if we are modeling molecules, we want to represent the belief that molecules are invariant to rotations. In modeling a physical system, we also want to represent approximate invariances, to account for friction, wind, or external forces. These priors are especially targeted at making predictions when there is a distribution shift — such as if we train a model to label images using one CT device, and then are asked to label images from a different device, which may introduce different artifacts.
The second part shows how we can combine these priors with data to form expressive posterior beliefs, through methods such as Hamiltonian Monte Carlo. A major component of this part involves new benchmarks, enabling us to understand how effectively scalable methods can infer posterior distributions, in order to make accurate predictions with reliable uncertainty estimates.
The third part is focused on applications, particularly in protein design and drug discovery, autonomous driving, and medical imaging. These are all applications where we have expressive prior beliefs, require high quality uncertainty estimates, and must be robust to distribution shifts.
To learn more about his research, please Andrew Gordon Wilson’s personal website.
