skip to main content
10.1145/2835966.2835970acmotherconferencesArticle/Chapter ViewAbstractPublication PagesindiahciConference Proceedingsconference-collections
research-article

A Protocol to Evaluate Virtual Keyboards for Indian Languages

Published:17 December 2015Publication History

ABSTRACT

We were asked to evaluate input mechanisms for touch-screen devices with the objective of standardising one of them for each of 14 major languages of India. For this purpose, we propose a protocol that consists of a 45-minute long training session, a 20-word first-time usability test, and a longitudinal test consisting of about 30 sessions, each of which required the user to type about 10 phrases 4 to 6 words long (a total of 300 phrases). The evaluation should be done with school children from standards 4th to 7th. The course of the evaluation may last 2-4 weeks for each user. To help follow the protocol over this long period and to collate the data, we offer a tool. Currently, we provide test corpora for Assamese, Bengali, Gujarati, Hindi, Kannada, Marathi, Odia, Tamil, Telugu, and Urdu. We have ensured that each corpus represents a mix of informal communication between people, popular phrases from films, songs, poetry and public discourse, and formal texts from school books and literature. We have tagged each phrase according to typing difficulty, phrase length, and memorability and age appropriateness. We evaluated the protocol through pilot tests with 206 users in Marathi, Gujarati, Hindi, Bengali, Odia, Assamese and Tamil. In this paper, we present the original protocol, the detailed findings from the Marathi pilots, and the proposed modifications to the protocol.

References

  1. Census 2001, http://en.wikipedia.org/wiki/List_of_languages_by_number_of_native_speakers_in_IndiaGoogle ScholarGoogle Scholar
  2. Jung, Y., Joshi, D., Narayanan-Saroja, V., and Desai, D. P. 2011. Solving the great Indian text input puzzle: touch screen-based mobile text input design. In Proceedings of the 13th International Conference on Human Computer Interaction with Mobile Devices and Services (Stockholm (2011) MobileHCI '11, ACM, New York, NY 313--322. DOI= http://dl.acm.org/citation.cfm?id=2037421 Google ScholarGoogle ScholarDigital LibraryDigital Library
  3. FIRSTPOST, http://www.firstpost.com/business/biztech/3g-boom-afoot-in-india-to-cover-90-of-a-grade-circles-by-2015-1618115.htmlGoogle ScholarGoogle Scholar
  4. Joshi, A., Ganu, A., Chand, A., Parmar, V., and Mathur, G. 2004. Keylekh: a keyboard for text entry in Indic scripts. In CHI'04 extended abstracts on Human factors in computing systems, (Vienna 2004) CHI EA '04. ACM, New York, NY, 928--942. DOI= http://dl.acm.org/citation.cfm?id=985950 Google ScholarGoogle ScholarDigital LibraryDigital Library
  5. Rathod, A. and Joshi, A.2002. A Dynamic Text Input scheme for phonetic scripts like Devanagari. 2002. In Proceedings of Development by Design. (Bangalore 2002)Google ScholarGoogle Scholar
  6. Mistry, P. and Nayak, N. 2007. AKSHAR: A mechanism for inputting Indic scripts on digital devices, (Hyderabad 2007) 18--20. USID,Google ScholarGoogle Scholar
  7. Ghosh, S. and Joshi, A. 2014. Text Entry in Indian Languages on Mobile: User Perspectives. In Proceedings of the India HCI 2014 Conference on Human Computer Interaction. (Delhi 2014) IndiaHCI '14. ACM, New York, NY, 55--63. DOI= http://dl.acm.org/citation.cfm?id=2676710 Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. Hind Keyboards, https://play.google.com/store/search?q=Hindi+keyboard&c=appsGoogle ScholarGoogle Scholar
  9. Joshi, M., Joshi, A. N., Emmadi, N., and Malsattar, N. 2014. Swarachakra keyboard for Indic scripts. In Proceedings of the 1st International Conference on Mobile Software Engineering and Systems, (Hyderabad 2014) MOBILESoft 2014. ACM, New York, NY, 5--6. DOI= http://dl.acm.org/citation.cfm?id=2593905 Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. Sharma, M. K. 2013. Word Prediction System with Virtual Keyboard for Text Entry in Hindi. In ACM Transactions on Asian Language Information Processing, (ACM 2013) ACM, New York, NY, Article No. 8. DOI= http://dl.acm.org/citation.cfm?id=2617590 Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. Hinkle, L., Brouillette, A., Jayakar, S., Gathings, L., Lezcano, and M., Kalita, J. 2013. Design and Evaluation of Soft Keyboards for Brahmic Scripts. In ACM Transactions on Asian Language Information Processing. (ACM 2013) ACM New York, NY, Article No. 6. DOI= http://dl.acm.org/citation.cfm?id=2461318 Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. Malsattar, N., Emmadi, N., and Joshi, M.2014. Testing the efficacy of an Indic script virtual keyboard: Swarachakra. In: Proceedings of the India HCI 2014 Conference on Human Computer Interaction, (Delhi 2014). IndiaHCI '14. ACM New York, 160--165. DOI= http://dl.acm.org/citation.cfm?id=2677203 Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. TDIL: Inscript Standard, http://www.tdil-dc.in/index.php?option=com_vertical&parentid=12Google ScholarGoogle Scholar
  14. MacKenzie, I. S. and Soukoreff, R. W. 2003. Phrase sets for evaluating text entry techniques. In: CHI'03 extended abstracts on Human factors in computing systems,(Florida 2003) CHI EA '03. ACM New York, 754--755. DOI= http://dl.acm.org/citation.cfm?id=765971 Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. Joshi, A., Dalvi, G., Joshi, M., Rashinkar, P., and Sarangdhar, A. 2011. Design and evaluation of Devanagari virtual keyboards for touch screen mobile phones. In Proceedings of the 13th In-ternational Conference on Human Computer Interaction with Mobile Devices and Services. (Stockholm 2011) MobileHCI'11. ACM New York, 323--332.DOI= http://dl.acm.org/citation.cfm?id=2037422 Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. Mitra, S.2003. Minimally invasive education: a progress report on the "hole-in-the-wall" experiments. In British Journal of Educational Technology, (2003) vol. 34, 367--371. DOI=http://onlinelibrary.wiley.com/doi/10.1111/1467-8535.00333/abstractGoogle ScholarGoogle ScholarCross RefCross Ref
  17. CDAC Inscript: Unified Virtual Keyboard for Indian Languages, http://www.cdac.in/index.aspx?id=dl_android_uvkilGoogle ScholarGoogle Scholar
  18. Swarachakra Marathi Keyboard, https://play.google.com/store/apps/details?id=iit.android.swarachakraMarathiGoogle ScholarGoogle Scholar
  19. Google Hindi input, https://play.google.com/store/apps/details?id=com.google.android.apps.inputmethod.hindiGoogle ScholarGoogle Scholar
  20. Angry Birds, https://play.google.com/store/apps/details?id=com.rovio.angrybirdsGoogle ScholarGoogle Scholar
  21. Census 2001, http://en.wikipedia.org/wiki/List_of_countries_by_number_of_Internet_usersGoogle ScholarGoogle Scholar
  22. Joshi, A., Dalvi, G., and Joshi, M. 2014. Corpus of Marathi Word Frequencies from Touch-Screen Devices Using Swarachakra Android Keyboard. In: Proceedings of the India HCI 2014 Conference on Human Computer Interaction, p. 74. ACM, Delhi (2014) Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. FIRE: Forum for Information Retrieval (FIRE), Information Retrieval Society of India. (12 2-4). Mumbai, Maharashtra, India, http://www.isical.ac.in/~fire/2011/index.html, 2011Google ScholarGoogle Scholar
  24. Baker, P., Hardie, A., McEnery, T., Xiao, R., Bontcheva, K., Cunningham, H., Gaizauskas, R., Hamza, O., Maynard, D., Tablan, V., Ursu, C., and Leisher, M. 2004. Corpus linguistics and South Asian languages: Corpus creation and tool development. Literary and Linguistic Computing, vol. 19, pp. 509--524. (2004)Google ScholarGoogle ScholarCross RefCross Ref
  25. Walmsley, W. S., Snelgrove, and W. X., Truong, K. N. 2014. Disambiguation of imprecise input with one-dimensional rotational text entry. ACM Transactions on Computer-Human Interaction, (2014) ACM New York, NY, vol. 21, 4. DOI=http://dl.acm.org/citation.cfm?id=2542544 Google ScholarGoogle ScholarDigital LibraryDigital Library
  26. Markussen, A., Jakobsen, M. R., and Hornbæk, K. 2014. Vulture: a mid-air word-gesture keyboard. In Proceedings of the 32nd annual ACM conference on Human factors in computing systems, (Toronto 2014) CHI '14, ACM, New York, NY 1073--1082. DOI= http://dl.acm.org/citation.cfm?id=2556964 Google ScholarGoogle ScholarDigital LibraryDigital Library
  27. Dearman, D., Karlson, A., and Meyers, B., Bederson, B. 2010. Multi-modal text entry and selection on a mobile device. In: Proceedings of Graphics Interface 2010, (Toronto 2010) GI '10, Canadian Information Processing Society Toronto, Ont., Canada, 19--26. DOI=http://dl.acm.org/citation.cfm?id=1839219 Google ScholarGoogle ScholarDigital LibraryDigital Library
  28. ASER, http://img.asercentre.org/docs/Publications/ASER%20Reports/ASER%202014/fullaser2014 mainreport.pdfGoogle ScholarGoogle Scholar
  29. Jones, E., Alexander, J., Andreou, A., Irani, P., and Subramanian, S. 2010. GesText: accelerometer-based gestural text-entry systems. In Proceedings of the SIGCHI Conference on Human Fac-tors in Computing Systems,(Atlanta 2010) CHI'10, ACM New York, NY, 2173--2182. DOI= http://dl.acm.org/citation.cfm?id=1753655 Google ScholarGoogle ScholarDigital LibraryDigital Library
  30. Fuccella, V., Isokoski, P., and Martin, B. 2013. Gestures and widgets: performance in text editing on multi-touch capable mobile devices. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, (Paris 2013) CHI '13, ACM New York, NY, 2785--2794. DOI= http://dl.acm.org/citation.cfm?id=2481385 Google ScholarGoogle ScholarDigital LibraryDigital Library
  31. Jain, M., Tekchandani, and K., Truong, K. N. 2013. Evaluating the Effect of Phrase Set in Hindi Text Entry. In Human-Computer Interaction--INTERACT 2013, (Heidelberg 2013). Interact-2013 195--202. DOI= http://www.dgp.toronto.edu/~mjain/INTERACT-2013a.pdfGoogle ScholarGoogle ScholarCross RefCross Ref
  32. Sarcar, S. and Panwar, P. 2013. Eyeboard++: an enhanced eye gaze-based text entry system in Hindi. In Proceedings of the 11th Asia Pacific Conference on Computer Human Interaction, (Bangalore 2013) APCHI '13, ACM New York, NY, 354--363. DOI=http://dl.acm.org/citation.cfm?id=2525304 Google ScholarGoogle ScholarDigital LibraryDigital Library
  33. Swype Marathi Keyboard, https://play.google.com/store/apps/details?id=com.nuance.swype.dtc&hl=enGoogle ScholarGoogle Scholar
  34. VKB App tool and corpus, goo.gl/1TsT5AGoogle ScholarGoogle Scholar

Recommendations

Comments

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Sign in
  • Published in

    cover image ACM Other conferences
    IndiaHCI '15: Proceedings of the 7th Indian Conference on Human-Computer Interaction
    December 2015
    182 pages
    ISBN:9781450340533
    DOI:10.1145/2835966

    Copyright © 2015 ACM

    © 2015 Association for Computing Machinery. ACM acknowledges that this contribution was authored or co-authored by an employee, contractor or affiliate of a national government. As such, the Government retains a nonexclusive, royalty-free right to publish or reproduce this article, or to allow others to do so, for Government purposes only.

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    • Published: 17 December 2015

    Permissions

    Request permissions about this article.

    Request Permissions

    Check for updates

    Qualifiers

    • research-article
    • Research
    • Refereed limited

    Acceptance Rates

    Overall Acceptance Rate33of93submissions,35%

PDF Format

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader