ShapeLearner: Towards Shape-Based Visual Knowledge Harvesting

The proliferation of images on the Web (a) enables us to extract shapes to train ShapeLearner (b), a 2D shape learning system that acquires knowledge of shape families, geometrical instances of their inner parts and their inter-relations. Given an unknown shape (c), the system automatically determines a classification, segmentation, and hierarchical part annotation (d).

The deluge of images on the Web has led to a number of efforts to organize images semantically and mine visual knowledge. Despite enormous progress on categorizing entire images or bounding boxes, only few studies have targeted fine-grained image understanding at the level of specific shape contours. For instance, beyond recognizing that an image portrays a cat, we may wish to distinguish its legs, head, tail, and so on. To this end, we present ShapeLearner, a system that acquires such visual knowledge about object shapes and their parts in a semantic taxonomy, and then is able to exploit this hierarchy in order to analyze new kinds of objects that it has not observed before. ShapeLearner jointly learns this knowledge from sets of segmented images. The space of label and segmentation hypotheses is pruned and then evaluated using Integer Linear Programming. Experiments on a variety of shape classes show the accuracy and effectiveness of our method.

Please cite our paper if you use the dataset.

Paper (2.12M)	comming soon	Binary Images(8376)	Result Images(2374)

We thank the reviewers for their valuable comments. This project was sponsored by National 973 Program (No.2015CB352500), National Natural Science Foundation of China (No. 61503217), Shandong Provincial Natural Science Foundation of China (No. ZR2014FP002), and The Fundamental Research Funds of Shandong University (No. 2014TB005, 2014JC001). Gerard de Melo’s research is supported by China 973 Program Grants 2011CBA00300, 2011CBA00301, and NSFC Grants 61033001, 61361136003, 61550110504.

@inproceedings{ShapeLearner2016,
author = {Huayong Xu and Yafang Wang and Kang Feng and Gerard de Melo and Wei Wu and Andrei Sharf and Baoquan Chen},
title = {ShapeLearner: Towards Shape-Based Visual Knowledge Harvesting},
booktitle = {Proceedings of ECAI 2016},
year = {2016},
pages = {435--443},
doi = {10.3233/978-1-61499-672-9-435},
editor = {Gal A. Kaminka and Maria Fox and Paolo Bouquet and EykeH{\"{u}}llermeier and Virginia Dignum and Frank Dignum and Frank van Harmelen},
series = {Frontiers in Artificial Intelligence and Applications},
volume = {285},
publisher = {{IOS} Press},
isbn = {978-1-61499-671-2},
}