October, 2018

Improving Active Learning by Avoiding Ambiguous Samples

Christian Limberg, Heiko Wersing, Helge Ritter

International Conference on Artificial Neural Networks (ICANN)

Abstract: If label information in a classification task is expensive, it can be beneficial to use active learning to get the most informative samples to label by a human. However, there can be samples which are meaningless to the human or recorded wrongly. If these samples are near the classifier’s decision boundary, they are queried repeatedly for labeling. This is inefficient for training because the human can not label these samples correctly and this may lower human acceptance. We introduce an approach to compensate the problem of ambiguous samples by excluding clustered samples from labeling. We compare this approach to other state-of-the-art methods. We further show that we can improve the accuracy in active learning and reduce the number of ambiguous samples queried while training.

April, 2018

Efficient Accuracy Estimation for Instance-Based Incremental Active Learning

Christian Limberg, Heiko Wersing, Helge Ritter

European Symposium on Artificial Neural Networks (ESANN)

Abstract: Estimating systems accuracy is crucial for applications of incremental learning. In this paper, we introduce the Distogram Estimation (DGE) approach to estimate the accuracy of instance-based classifiers. By calculating relative distances to samples it is possible to train an offline regression model, capable of predicting the classifiers accuracy on unseen data. Our approach requires only a few supervised samples for training and can instantaneously be applied on unseen data afterwards. We evaluate our method on five benchmark data sets and for a robot object recognition task. Our algorithm clearly outperforms two baseline methods both for random and active selection of incremental training examples.