Paul Schmieder
Beryl Plimmer
ABSTRACT
We present our toolkit to automatically evaluate recognition algorithms. There are few published comparative evaluations of sketch recognition algorithms and those that exist do not provide benchmarking or direct comparisons because standardised data and an evaluation platform is not available. By unifying data collection, labelling and evaluation in one tool, fair, flexible and comprehensive evaluations are possible. Currently we have 6 existing recognizers integrated into this tool. With our initial evaluations of these recognizers we have observed that the context from which training data is taken has an effect on recognition success rates. These results suggest that an evaluation platform such as this is a powerful adjunct for sketch recognition research.
- {BPGW08} Blagojevic R., Plimmer B., Grundy J., Wang Y.: A Data Collection Tool for Sketched Diagrams, 5th Eurographics Conference on Sketch Based Interfaces and Modelling (SBIM '08), Annecy, France. Google Scholar
Digital Library
- {BSP09} Blagojevic R., Schmieder P., Plimmer B.: Towards a Toolkit for the Development and Evaluation of Sketch Recognition Techniques. Intelligent User Interfaces (IUI '09), Florida, USA.Google Scholar
- {FPJ02} Fonseca M. J., Pimentel C. E., Jorge J. A.: CALI: An Online Scribble Recogniser for Calligraphic Interfaces. AAAI Spring Symposium on Sketch Understanding.Google Scholar
- {PPGI07} Patel R., Plimmer B., Grundy J., Ihaka R.: Ink Features for Diagram Recognition. 4th Eurographics Workshop on Sketch-Based Interfaces and Modeling Riverside, California: 131--138. Google ScholarDigital Library
- {PH08} Paulson, B., Hammond T.: PaleoSketch: accurate primitive sketch recognition and beautification, Proceedings of the 13th international conference on Intelligent user interfaces. ACM, Gran Canaria, Spain. Google ScholarDigital Library
- {PWJH08} Paulson B., Wolin A., Johnston J., Hammond T.: SOUSA: Sketch-based Online User Study Applet. Sketch Based Interfaces and Modeling, Annecy, France: 81--88. Google ScholarDigital Library
- {PF07} Plimmer B., Freeman I.: A Toolkit Approach to Sketched Diagram Recognition. HCI, Lancaster, UK, 1: 205--213. Google ScholarDigital Library
- {Rub91} Rubine D. H.: Specifying gestures by example. Proceedings of Siggraph '91: 329--337. Google ScholarDigital Library
- {SNK07} Signer B., Norrie M. C., Kurmann U.: iGesture: A Java Framework for the Development and Deployment of Stroke-Based Online Gesture Recognition Algorithms.Google Scholar
- {WF02} Witten I. H., Frank E.: Data mining: practical machine learning tools and techniques with Java implementations. ACM SIGMOD Record 31: 76--77. Google ScholarDigital Library
- {WWL07} Wobbrock J. O., Wilson A. D., Li Y.: Gestures without libraries, toolkits or training: a $1 recognizer for user interface prototypes, Proceedings of the 20th annual ACM symposium on User interface software and technology. ACM, Newport, Rhode Island, USA. Google ScholarDigital Library
- {WSA07} Wolin A., Smith D., Alvarado C.: A Pen-based Tool for Efficient Labelling of 2D Sketches. 4th Eurographics Workshop on Sketch-Based Interfaces and Modeling, Riverside, CA. Google ScholarDigital Library
Index Terms
- Automatic evaluation of sketch recognizers
Recommendations
Evaluation on Selection Criteria of Multiple Numeral Recognizers with the Fixed Number of Recognizers
ICPR '02: Proceedings of the 16 th International Conference on Pattern Recognition (ICPR'02) Volume 3 - Volume 3The combination of multiple recognizers shows various performances according to the selection of recognizers and the combination method of them. The selection of recognizers on how to select them or how many to select them still remains one of important ...
Face Photo-Sketch Synthesis and Recognition
In this paper, we propose a novel face photo-sketch synthesis and recognition method using a multiscale Markov Random Fields (MRF) model. Our system has three components: 1) given a face photo, synthesizing a sketch drawing; 2) given a face sketch ...
Sketch-Based Evaluation of Image Segmentation Methods
A method for evaluating image segmentation methods is proposed in this paper. The method is based on a perception model where the drawing act is used to represent visual mental percepts. Each segmented image is represented by a minimal set of features ...
Comments