Reading Deep One-class Classification

 Deep One-class Classification: a two-stage framework for deep one-class classification.


1) learn self-supervised representations from one-class data

2) build one-class classifiers on learned representations

Self-supervised representation algorithms, the connections to existing one-class classification methods, SOTA contrastive representation learning for one-class classification


A: the stochastic data augmentation process, including resize and crop, horizontal flip, color jittering, gray-scale and gaussian blur.

Feature extractor f, loss L, projection head g.

network structure as gof, g is the projection head used to compute proxy losses and f outputs representations for the downstream task.


One way of representation learning is to learn by discriminating augmentations applied to the data. Although not trained to do so, the likelihood of learned rotation classifiers is shown to well approximate the normality score and has been used for one-class classification. A plausible explanation is via outlier exposure, where the classifier learns a decision boundary distinguishing original images from simulated outliers by image rotation. 


Contrastive learning learns representation by distinguishing different views of itself from other data instances. It could be problematic for one-class classification. 

1. a class collision. The contrastive loss is minimized by maximizing the distance between representations of negative pairs even though they are from the same class when applied to the one-class classification. 

2. a uniformity of representations. Reducing the uniformity is easier to isolate outliers from inliers.


One-Class Contrastive Learning: 

1) to reduce the uniformity of representations, a moderate M (batch size) is used but not large M.

2) distribution augmentation is proposed for one-class contrastive learning. Instead of modeling the training data distribution, model the union of augmented training distribution; employ geometric transformations, such as rotation or horizontal flip, for distribution augmentation.


To construct a classifier, for generative approaches, use nonparametric kernel density estimation (KDE) to estimate densities from learned representations; for discriminative approaches, train one-class SVMs.


Experiments:

Evaluate on one-class classification benchmarks, including CIFAR-10, CIFAR-100, Fashion-MNIST, and Cat-vs-Dog. Images from one class are given as inlier and those from remaining classes are given as outlier. 

CelebA eyeglasses dataset, face images with eyeglasses are denoted as an outliers. Lastly, in addition to semantic anomaly detection, also consider the defect detection benchmark MVTec.


Results:

Report the mean and standard deviation of AUCs averaged over classes over 5 runs. The mean of 5 datasets is weighted by the number of classes for each dataset. 



Paper source: https://openreview.net/pdf?id=HCSgyPUfeDj








Comments

Post a Comment

Popular posts from this blog

Reading CLIP

Learning directly from raw text about images is promising which leverages a much broader source of supervision. Task: predicting which caption goes with which image is an efficient and scalable way to learn SOTA image representations from scratch on a dataset of pairs. After pre-training, natural language is used to is used to reference learned visual concepts enabling zero-shot transfer of the model to downstream tasks.   [Paper Source]  https://cdn.openai.com/papers/Learning_Transferable_Visual_Models_From_Natural_Language.pdf Radford, A., Kim, J.W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J. and Krueger, G., 2021. Learning transferable visual models from natural language supervision.  arXiv preprint arXiv:2103.00020 .
Read more

Reading CutPaste

 CutPaste: a simple data augmentation strategy that cuts an image patch and pastes at a random location of a large image. Dataset: MVTec anomaly detection dataset Aim: detect various types of real-world defects [ Code ] Paper Source:  https://openaccess.thecvf.com/content/CVPR2021/papers/Li_CutPaste_Self-Supervised_Learning_for_Anomaly_Detection_and_Localization_CVPR_2021_paper.pdf
Read more

OOD-related papers

Deep One-Class Classification, ICLR 2021 [ Paper ] [ Code ] Google AI [ Blog ] [ Code ] Learning from Failure: Training Debiased Classifier from Biased Classifier [ Paper ] [ Code ] NeurIPS 2020 Learning Debiased Representation via Disentangled Feature Augmentation [ Paper ] [ Code ] NeurIPS 2021 Graph Convolution for Semi-Supervised Classification: Improved Linear Separability and Out-of-Distribution Generalization [ Paper ] Explainable Deep One-class Classification [ Paper ] GAN Ensemble for Anomaly Detection [ Paper ] Neural Transformation Learning for Deep Anomaly Detection Beyond Images [ Paper ] Learning Semantic Context from Normal Samples for Unsupervised Anomaly Detection Dual Compositional Learning in Interactive Image Retrieval [*] SSL  Drop, Swap, and Generate: A Self-Supervised Approach for Generating Neural Activity [ Paper ] [ Code ] NeurIPS 2021 Unsupervised Learning of Discriminative Attributes and Visual Representations [ Paper ] CVPR2016 Common Visual Pattern Dis...
Read more