Mathematical Sciences Research Institute

Home » Workshop » Schedules » Predicting Out-of-Distribution Error with the Projection Norm

Predicting Out-of-Distribution Error with the Projection Norm

[Virtual] Hot Topics: Foundations of Stable, Generalizable and Transferable Statistical Learning March 07, 2022 - March 10, 2022

March 10, 2022 (10:00 AM PST - 10:25 AM PST)
Speaker(s): Jacob Steinhardt (UC Berkeley)
Location: MSRI: Online/Virtual
Primary Mathematics Subject Classification No Primary AMS MSC
Secondary Mathematics Subject Classification No Secondary AMS MSC

Predicting Out-Of-Distribution Error With The Projection Norm


We will consider a metric---the "Projection Norm"---that predicts a model's performance on out-of-distribution (OOD) data, without access to ground truth labels. Projection Norm first uses model predictions to pseudo-label test samples and then trains a new model on the pseudo-labels. The more the new model's parameters differ from an in-distribution model, the greater the predicted OOD error. Empirically, this outperforms existing methods on both image and text classification tasks and across different network architectures. Theoretically, we connect our approach to a bound on the test error for overparameterized linear models. Furthermore, we find that Projection Norm is the only approach that achieves non-trivial detection performance on adversarial examples.

Joint work with Yaodong Yu, Zitong Yang, Alex Wei, and Yi Ma. https://arxiv.org/abs/2202.05834

Supplements No Notes/Supplements Uploaded
Video/Audio Files

Predicting Out-Of-Distribution Error With The Projection Norm

Troubles with video?

Please report video problems to itsupport@msri.org.

See more of our Streaming videos on our main VMath Videos page.