Affiliation of Author(s):计算机科学与技术学院/人工智能学院/软件学院
Journal:IJCAI Int. Joint Conf. Artif. Intell.
Abstract:In traditional active learning, there is only one labeler that always returns the ground truth of queried labels. However, in many applications, multiple labelers are available to offer diverse qualities of labeling with different costs. In this paper, we perform active selection on both instances and labelers, aiming to improve the classification model most with the lowest cost. While the cost of a labeler is proportional to its overall labeling quality, we also observe that different labelers usually have diverse expertise, and thus it is likely that labelers with a low overall quality can provide accurate labels on some specific instances. Based on this fact, we propose a novel active selection criterion to evaluate the cost-effectiveness of instance-labeler pairs, which ensures that the selected instance is helpful for improving the classification model, and meanwhile the selected labeler can provide an accurate label for the instance with a relative low cost. Experiments on both UCI and real crowdsourcing data sets demonstrate the superiority of our proposed approach on selecting cost-effective queries.
ISSN No.:1045-0823
Translation or Not:no
Date of Publication:2017-01-01
Co-author:Chen, Jia-Lve,Mu, Xin,Zhou, Zhi-Hua
Correspondence Author:黄圣君,Chen, Jia-Lve,Sheng Jun Huang
Date of Publication:2017-01-01