Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics

Vol. 1: Long Papers. Association for Computational Linguistics, 2022.

Uncertainty estimation (UE) of model predictions is a crucial step for a variety of tasks such as active learning, misclassification detection, adversarial attack detection, out-of-distribution detection, etc. Most of the works on modeling the uncertainty of deep neural networks evaluate these methods on image classification tasks. Little attention has been paid to UE in natural language processing. To fill this gap, we perform a vast empirical investigation of state-of-the-art UE methods for Transformer models on misclassification detection in named entity recognition and text classification tasks and propose two computationally efficient modifications, one of which approaches or even outperforms computationally intensive methods.

Chapters

Uncertainty Estimation of Transformer Predictions for Misclassification Detection

Vazhentsev A., Kuzmin G., Shelmanov A. et al., , in: Proceedings of the 60th Annual Meeting of the Association for Computational LinguisticsVol. 1: Long Papers.: Association for Computational Linguistics, 2022. P. 8237–8252.

Uncertainty estimation (UE) of model predictions is a crucial step for a variety of tasks such as active learning, misclassification detection, adversarial attack detection, out-of-distribution detection, etc. Most of the works on modeling the uncertainty of deep neural networks evaluate these methods on image classification tasks. Little attention has been paid to UE in natural ...

Added: December 13, 2022

Keywords: natural language processing, Uncertainty Estimation, transformers, natural language processing with deep neural networks