The problem of novelty detection in fine-grained visual classification (FGVC) is considered. An integrated understanding of the probabilistic and distance-based approaches to novelty detection is developed within the framework of convolutional neural networks (CNNs). It is shown that softmax CNN classifiers are inconsistent with novelty detection, because their learned class-conditional distributions and associated distance metrics are unidentifiable. A new regularization constraint, the class-conditional Gaussianity loss, is then proposed to eliminate this unidentifiability, and enforce Gaussian class-conditional distributions. This enables training Novelty Detection Consistent Classifiers (NDCCs) that are jointly optimal for classification and novelty detection. Empirical evaluations show that NDCCs achieve significant improvements over the state-of-the-art on both small- and large-scale FGVC datasets.

Paper

---

Published in International Conference on Computer Vision (ICCV), 2023.

Supplement

Arxiv

Dataset/Code

Bibtex

Models

---

Illustration of the pseudo UQ and RoI Mixup. The right table shows the label for different visual question inputs

Code

Training, evaluation and deployment code available on GitHub.

Video

---