Vidya Setlur
Melanie Tory
ABSTRACT
Natural language interfaces for visualizations have emerged as a promising new way of interacting with data and performing analytics. Many of these systems have fundamental limitations. Most return minimally interactive visualizations in response to queries and often require experts to perform modeling for a set of predicted user queries before the systems are effective. Eviza provides a natural language interface for an interactive query dialog with an existing visualization rather than starting from a blank sheet and asking closed-ended questions that return a single text answer or static visualization. The system employs a probabilistic grammar based approach with predefined rules that are dynamically updated based on the data from the visualization, as opposed to computationally intensive deep learning or knowledge based approaches.
The result of an interaction is a change to the view (e.g., filtering, navigation, selection) providing graphical answers and ambiguity widgets to handle ambiguous queries and system defaults. There is also rich domain awareness of time, space, and quantitative reasoning built in, and linking into existing knowledge bases for additional semantics. Eviza also supports pragmatics and exploring multi-modal interactions to help enhance the expressiveness of how users can ask questions about their data during the flow of visual analysis.
Supplemental Material
Available for Download
- Extended Backus-Naur Form. https://en.wikipedia.org/ wiki/Extended_Backus%E2%80%93Naur_Form.Google Scholar
- I-want-to-go moments: From search to store. https://www.thinkwithgoogle.com/articles/i-want-to-go-micro-moments.html.Google Scholar
- IBM Watson Analytics. http://www.ibm.com/analytics/watson-analytics/.Google Scholar
- Microsoft Q & A. https://powerbi.microsoft.com/en-us/documentation/powerbi-service-q-and-a/.Google Scholar
- NarrativeScience. https://www.narrativescience.com/quill.Google Scholar
- Node.js®. https://nodejs.org/.Google Scholar
- ThoughtSpot. http://www.thoughtspot.com/.Google Scholar
- Turf: Advanced geospatial analysis for browsers and node. http://turfjs.org.Google Scholar
- WolframAlpha. https://www.wolframalpha.com/.Google Scholar
- Allen, J. Recognizing Intentions from Natural Language Utterances. In Computational Models of Discourse, M. Brady, Ed. M.I.T. Press, Cambridge, Massachusetts, 1982.Google Scholar
- Androutsopoulos, I., Ritchie, G. D., and Thanisch, P. Natural language interfaces to databases-an introduction. Natural Language Engineering 1, 1 (1995), 29--81. Google ScholarCross Ref
- Aurisano, J., Kumar, A., Gonzales, A., Reda, K., Leigh, J., Di Eugenio, B., and Johnson, A. Show me data' observational study of a conversational interface in visual data exploration. IEEE VIS (2015).Google Scholar
- Bostock, M., Ogievetsky, V., and Heer, J. D3: Data-driven documents. IEEE Transactions on Visualization & Computer Graphics (Proc. InfoVis) (2011). Google ScholarDigital Library
- Carbonell, J. G., Boggs, W. M., Mauldin, M. L., and Anick, P. G. The xcalibur project, a natural language interface to expert systems and data bases. Proceedings of the Eighth International Joint Conference on Artificial Intelligence (1985). Google ScholarDigital Library
- Cover, T. M., and Thomas, J. A. Elements of Information Theory. Wiley-Interscience, New York, NY, USA, 1991. Google ScholarDigital Library
- Cox, K., Grinter, R. E., Hibino, S. L., Jagadeesan, L. J., and Mantilla, D. A multi-modal natural language interface to an information visualization environment. International Journal of Speech Technology 4, 3 (2001), 297--314. Google ScholarCross Ref
- Egenhofer, M. Spatial sql: A query and presentation language. IEEE Transactions on Knowledge and Data Engineering 6, 1 (1994), 86--95. Google ScholarDigital Library
- Finin, T., Joshi, A. K., and Webber, B. Natural language interactions with artificial experts. Proceedings of the IEEE 74, 7 (June 1986), 921--938. Google ScholarCross Ref
- Frank, A. U., and Mark, D. M. Language issues for geographical information systems. In Geographical Information Systems: Principles and Applications, vol 1, D. Maguire, M. Goodchild, and D. Rhind, Eds. Longman, London, 1991, 147--153.Google Scholar
- Freeman, J. The modelling of spatial relations. Computer Graphics and Image Processing 4, 2 (1975), 156--171. Google ScholarCross Ref
- Gao, T., Dontcheva, M., Adar, E., Liu, Z., and Karahalios, K. G. Datatone: Managing ambiguity in natural language interfaces for data visualization. In Proceedings of the 28th Annual ACM Symposium on User Interface Software Technology, UIST '15, ACM (New York, NY, USA, 2015), 489--500. Google ScholarDigital Library
- Grammel, L., Tory, M., and Storey, M. A. How information visualization novices construct visualizations. IEEE Transactions on Visualization and Computer Graphics 16, 6 (Nov 2010), 943--952. Google ScholarDigital Library
- Li, F., and Jagadish, H. V. Constructing an interactive natural language interface for relational databases. Proc. VLDB Endow. 8, 1 (Sept. 2014), 73--84. Google ScholarDigital Library
- Montello, D., Goodchild, M., Gottsegen, J., and Fohl, P. Where's downtown' behavioral methods for determining referents for vague spatial queries. Spatial Cognition and Computation 3, 2&3 (2003), 185--204. Google ScholarCross Ref
- Ng, H. T., and Zelle, J. Corpus-based approaches to semantic interpretation in natural language processing. AI Magazine Winter 1997, 45--64 (1997).Google Scholar
- Oviatt, S., and Cohen, P. Perceptual user interfaces: Multimodal interfaces that process what comes naturally. Commun. ACM 43, 3 (Mar. 2000), 45--53. Google ScholarDigital Library
- Parr, T. The Definitive ANTLR 4 Reference, 2nd ed. Pragmatic Bookshelf, 2013. Google ScholarDigital Library
- Pedersen, T., Patwardhan, S., and Michelizzi, J. Wordnet::similarity: Measuring the relatedness of concepts. In Demonstration Papers at HLT-NAACL 2004, HLT-NAACL-Demonstrations '04, Association for Computational Linguistics (Stroudsburg, PA, USA, 2004), 38--41. Google ScholarDigital Library
- Popescu, A.-M., Etzioni, O., and Kautz, H. Towards a theory of natural language interfaces to databases. In Proceedings of the 8th International Conference on Intelligent User Interfaces, IUI '03, ACM (New York, NY, USA, 2003), 327--327. Google ScholarDigital Library
- Pustejovsky, J., Castaño, J., Ingria, R., Saurí, R., Gaizauskas, R., Setzer, A., and Katz, G. Timeml: Robust specification of event and temporal expressions in text. In in Fifth International Workshop on Computational Semantics (IWCS-5 (2003).Google Scholar
- Reinhart, T. Pragmatics and Linguistics: An Analysis of Sentence Topics. IU Linguistics Club publications. Reproduced by the Indiana University Linguistics Club, 1982.Google Scholar
- Roche, E., and Shabes, Y., Eds. Finite-State Language Processing. MIT Press, Cambridge, MA, USA, 1997. Google ScholarDigital Library
- Salton, G., and McGill, M. J. Introduction to Modern Information Retrieval. McGraw-Hill, Inc., New York, NY, USA, 1986. Google ScholarDigital Library
- Sun, Y., Leigh, J., Johnson, A., and Lee, S. Articulate: A Semi-automated Model for Translating Natural Language Queries into Meaningful Visualizations. Springer Berlin Heidelberg, 2010, 184--195. Google ScholarDigital Library
- Talmy, L. How language structures space. In Spatial Orientation: Theory, Research, and Application, H. Pick and L. Acredolo, Eds. Plenum, New York, 1983. Google ScholarCross Ref
- Winograd, T. Procedures as a Representation for Data in a Computer Program for Understanding Natural Language. PhD thesis, 1971.Google Scholar
- Wu, Z., and Palmer, M. Verbs semantics and lexical selection. In Proceedings of the 32nd Annual Meeting on Association for Computational Linguistics, ACL '94, Association for Computational Linguistics (Stroudsburg, PA, USA, 1994), 133--138. Google ScholarDigital Library
Index Terms
- Eviza: A Natural Language Interface for Visual Analysis
Recommendations
An interrogative visualization environment for large-scale engineering simulations
An interrogative visualization environment is described for the interactive display and querying of large datasets. The environment combines a web-based intelligent agent facility with a visualization engine. The intelligent agent facility (IAF) ...
Hierarchical visual filtering, pragmatic and epistemic actions for database visualization
SAC '13: Proceedings of the 28th Annual ACM Symposium on Applied ComputingVisualization techniques of all sorts suffer from visual cluttering, the occlusion of visual information due to the overlap of graphical items; and from excessive complexity in analytical tasks due to multiple parallel perspectives. To cope with these ...
Understanding conversational interaction in multiparty conversations: the EVA Corpus
AbstractThis paper focuses on gaining new knowledge through observation, qualitative analytics, and cross-modal fusion of rich multi-layered conversational features expressed during multiparty discourse. The outlined research stems from the theory that ...
Comments