We propose to improve BERT model calibration via on-manifold smoothing and off-manifold smoothing.

Reference

  • Calibrated Fine-Tuning for Pre-trained Language Models via Manifold Smoothing 2020
    Lingkai Kong, Haoming Jiang, Yuchen Zhuang, Jie Lyu, Tuo Zhao and Chao Zhang [arXiv]
    Conference on Empirical Methods in Natural Language Processing (EMNLP), 2020