Reverse KL-Divergence Training of Prior Networks: Improved Uncertainty and Adversarial Robustness

Malinin A.; Gales M.

Publications

?

Reverse KL-Divergence Training of Prior Networks: Improved Uncertainty and Adversarial Robustness

Malinin A., Gales M.

Language: English

Keywords: Uncertainty Estimation

In book

Advances in Neural Information Processing Systems 32 (NeurIPS 2019)

[б.и.], 2019.

LM-Polygraph: Uncertainty Estimation for Language Models

Fadeeva E., Vashurin R., Tsvigun A. et al., , in: Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing.: Singapore: Association for Computational Linguistics, 2023. P. 446 –461.

Recent advancements in the capabilities of large language models (LLMs) have paved the way for a myriad of groundbreaking applications in various fields. However, a significant challenge arises as these models often “hallucinate”, i.e., fabricate facts without providing users an apparent means to discern the veracity of their statements. Uncertainty estimation (UE) methods are one ...

Added: February 17, 2025

Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics

Association for Computational Linguistics, 2022.

Uncertainty estimation (UE) of model predictions is a crucial step for a variety of tasks such as active learning, misclassification detection, adversarial attack detection, out-of-distribution detection, etc. Most of the works on modeling the uncertainty of deep neural networks evaluate these methods on image classification tasks. Little attention has been paid to UE in natural ...

Added: May 17, 2022

Ensemble Distribution Distillation

Malinin A., Mlodozeniec B., Gales M., , in: Proceedings of the 8th International Conference on Learning Representations (ICLR 2020).: ICLR, 2020.

Added: November 1, 2021

Uncertainty Estimation in Autoregressive Structured Prediction

Andrey Malinin, Gales M., , in: Proceedings of the 9th International Conference on Learning Representations (ICLR 2021). ICLR, 2021.: ICLR, 2021. P. 1–31.

Added: November 1, 2021

Scaling Ensemble Distribution Distillation to Many Classes with Proxy Targets

Ryabinin M., Malinin A., Gales M., , in: Advances in Neural Information Processing Systems 34 (NeurIPS 2021).: Curran Associates, Inc., 2021. P. 6023–6035.

Added: October 31, 2021

Uncertainty Estimation via Stochastic Batch Normalization

Ashukha A., Vetrov D., Molchanov D. et al., , in: Workshop of the 6th International Conference on Learning Representations (ICLR).: International Conference on Learning Representations, ICLR, 2018. P. 1–6.

In this work, we investigate Batch Normalization technique and propose its probabilistic interpretation. We propose a probabilistic model and show that Batch Normalization maximazes the lower bound of its marginalized log-likelihood. Then, according to the new probabilistic model, we design an algorithm which acts consistently during train and test. However, inference becomes computationally inefficient. To ...

Added: October 31, 2018