Keywords

1 Introduction

Reading is an activity needed in most facets of our daily life. By reading, people gather information from text, extract meaning, build knowledge and foster other cognitive abilities, such as reasoning [38]. The information age changes our reading behavior. With an abundance of information made available through the Internet and accessible on electronic devices, reading becomes a ubiquitous activity and the overall time spent reading increases [29]. However, the activity of reading on computers and other electronic devices tends to be shallower than on conventional media [12] and the skill to quickly scan a document in order to get the gist of its content becomes important.

Visualizations can be used to make readers quickly filter out the essence of a text and help with the decision on whether a text is worth reading in the first place [28]. Understanding the gist of a text supports overall text comprehension and allows readers to process a text’s content and meaning in more detail. By facilitating text reading in this way, people could manage their daily readings more effectively by reducing overall reading time or increasing reading volume.

In this work, we investigate the use of text visualizations to support reading activities on electronic devices with a special focus on effective gist extraction, comprehension, and perceived usefulness. Here, we make use of the Priming Effect – a concept taken from psychology that describes an implicit memory effect: by exposing users to a current stimulus, a response to another stimulus is triggered in the future [4]. We base our use of the term on the more conceptual definition of how the exposure to one stimulus influences the perception of a subsequent one. While this is barely a recognition task in the range of milliseconds, we use it to investigate an effect in higher-level cognitive tasks by applying the effect to reading. Therefore, we launched an investigation into the utility and effects of four different text visualizations: word cloud, highlighting, mind map, and image collage. After having conducted an initial focus group, we collected both quantitative and qualitative results in a comprehensive user study with 50 participants. Results show the feasibility of using mind maps and text highlights as primers. Based on our findings we derive requirements for building automated text visualizations for reading on electronic devices. The contribution of our work is as follows:

  1. 1.

    We show that text visualizations can be used to get the gist of a text prior to reading.

  2. 2.

    We compare four visualization types with regard to their utility, finding that mind maps are most useful in aiding understanding.

  3. 3.

    We provide insights into how highlighting can help to reduce reading time by giving readers a better text understanding and to go through text quicker.

  4. 4.

    We present an architecture for an automated system to support readers on electronic reading devices through visualizations.

2 Background and Related Work

Our research combines aspects of different research fields. In this section, we provide same insights into reading behavior and how it has changed as well as reading comprehension. Furthermore, we introduce the priming effect and provide information about text summaries and visualizations.

2.1 Reading Behavior

Cunningham and Stanovich [38] speak of the Matthew’s Effect when describing a ’rich get richer, poor get poorer’ - phenomenon for reading. The ability to comprehend and read large volumes creates a reciprocal relationship: readers with a broader knowledge base carry out a smaller amount of effort while gaining a more positive experience than readers who struggle to comprehend text [10]. Positive experiences can be a great motivation which puts avid readers into a positive feedback loop, which eventually affects vocabulary development, abilities of expression, and reasoning. Liu [29] investigated the effects the information age has had on people’s reading behavior: with more information constantly available, the time spent on reading increased, specifically the amount of work related reading. On the other hand, reading patterns get more shallow and fragmented as people read in less depth. This seems to be a coping mechanism through which people get through the high amount of daily readings and only read a text in detail if it is assumed to be highly relevant. The ability to scan and browse text for pointers to valuable information becomes increasingly important.

2.2 Reading Comprehension and Recall

Reading comprehension is essential to extract meaning from text. Comprehending a text is a combination of recalling what has been read and the understanding of the underlying concepts [31]. Therefore, the text needs to be interacted with on a mental level and connected to the reader’s existing knowledge of the world [1]. Being aware of the context instead of just the meaning of single words is also crucial for understanding. Federmeier and Kutas [17] showed that words are often predictable in sentence contexts and that sentence contexts can facilitate comprehension. Bransford and Johnson [8] investigated the role of contextual knowledge where comprehension often results from the context. In their experiments, they showed that participants who had to comprehend a text passage without being provided with the appropriate context did worse in comprehension tests than those who had been provided with the context. Another result of their experiments is that prior knowledge alone is not always helpful for the comprehension process but that it has to be activated first through some stimulus. Text comprehension and memory are deeply intertwined. Kintsch [24] explicitly contrasted remembering and learning, where remembering entails the process of simply reproducing the information of a text. This requires a shallower form of comprehension, whereas if a reader has effectively learned from a text, she can apply that knowledge to new situations. Reading comprehension is therefore a multi-faceted measure, which often is assessed by the reader’s ability to recall what is stated in the text (literal comprehension), interpret the text’s meaning by connecting rather implicitly stated information (inferential comprehension), or to relate the text’s content to prior experiences and knowledge (evaluative comprehension) [7]. Learning has occurred, if a reader has been able to build a well-structured representation of the text and connect this representation to the background knowledge she possesses [30].

2.3 The Priming Effect

When a stimulus subconsciously influences the processing of the following stimulus, psychologists speak of a Priming Effect. Influencing stimuli, the so-called primes, can have an effect on memory processing tasks [4]. Research in psychology has confirmed this effect for reading activities. Typical implicit memory testing methods include word primes, such as, for example for the prime ‘apple’ [33]: for word stem completion, such as A P _ _ _, for word fragment completion: _ P _ L _, for anagram solving: ’plape’, and word identification tasks.

Rajaram and Roediger [33] apply these tasks with different primes in different modalities (auditory or visual priming), forms (word or picture) and typefaces to evaluate the magnitude of priming effects. Their findings show that for modalities showing visual primes for visual tasks and auditory primes for auditory tasks is recommendable. Hence, we were interested in whether this effect can be used when using text summary visualizations as primers before reading a text to foster text comprehension and recall.

2.4 Text Summaries

Related work about text summaries belongs to the field of text mining. Text mining, in general, describes the process of extracting high-quality information from documents (see [22] for an overview). In this work, we focus on extracting keywords from text and make use of the term-frequency-inverse document frequency (tf-idf) method [34]. This method determines the relevance for each term by combining the term frequency (tf) of a term (a term with a high frequency in a document is probably more important than a term with a low frequency) with the inverse document frequency (a term that has a high frequency over all documents is less descriptive for a particular document). For example, the frequency of the term “and” is high in a single document as well as in a set of documents. Therefore, the tf-idf score is quite low. Whereas, the term “priming” has a high frequency in one document but not in the set of specific documents the tf-idf score is high and indicates an importance of this term. To reduce the impact of larger tf-idf values, a log-scale is applied:

$$ tf-idf_{t,d} \; = \; tf_{t,d} \times log(\frac{N}{df_t}) $$

Here, t stands for the term under investigation, d for the document in a corpus t belongs to. N depicts the total number of documents in the corpus which is divided by the number of documents containing the term \(df_t\).

2.5 Text Visualization

Banas and Brown [5] define visualization as “the conversion of information to a symbolic representation of a particular idea, concept, or data object”. Hence, representations can reveal patterns, relationships, and concepts helping observers to new understandings and deeper comprehension of text. Text visualizations also convey information quickly and efficiently and generally aim to make the comprehension of aspects easier. Kalmane [23] stresses the potential of online visualization tools to assist reading comprehension, especially for language learners. Adding context-relevant images, for example, helps understanding and learning overall [11]. In traditional textbooks, visualizations were often crafted for illustrative purposes, which were rather laborious to create. Now it is pragmatical to generate visualizations automatically and support readers in-situ. A lot of research has been carried out to automatically generate text visualizations. The complexity of these text visualizations ranges from visualizing single documents (see, e.g., [21, 40]) over visualizing the context of a book (see, e.g., [14, 32]) to visualizing documents and their relationships in digital libraries (see, e.g., [42]). Comprehensive overviews of different text visualization techniques are provided by Alencar et al. [2] or Kucher and Kerren [25]. In our work, we focus on single document visualizations based on their relation to a corpus of related documents.

3 Visualizations for Text Priming

We decided to investigate four different types of text visualizations with different characteristics regarding their effects on reading activities: word cloud (representing all extracted words), highlighting (representing keywords in their context), mind map (representing selected keywords and their connections to each other), and image collage (representing a couple of keywords in a more abstract form). We chose these common visualization types as they are rather well-established representations of texts. With regard to our user study, we thereby attempted to minimize possible novelty effects. In the following section, we describe these four visualization types, discuss why we think that they are worth considering for invigorating priming effects and how we designed them for our user study. In all cases, we build upon the 50 most relevant keywords extracted from a text through tf-idf.

3.1 Word Cloud

A set of visually presented words arranged in a cloud is commonly known as a “tag cloud”. If the content of a text is summarized, it is also often called a “word cloud”, which is the term we will use throughout this paper. Viegas and Wattenberg [41] provide a comprehensive overview of the history and different types of tag or word clouds. Such word clouds have been widely used for teaching and learning (see, e.g., [6]), which is why we are interested in their feasibility to prime a reader for a text. We apply a simple design for visualizing the 50 most descriptive keywords of a text by varying font sizes according to keyword’s importance and a color scheme that does not distract the reader but fosters readability. In our design, size and color reflect the importance of a word. The sequential word order of the keywords aims to mirror the text’s structure. Figure 1A shows a word cloud, which we used in our study.

Fig. 1.
figure 1

Visualizations used in our user study summarizing a text about learning.

Full size image

3.2 Highlighting

Printed text often contains in-text highlights in the form of keywords marked by being given a background color. Different studies showed the positive effects highlighting has on text retention (see, e.g., [18, 19]), whereby actively highlighting, performed by the reader herself during reading is considered more effective than pre-existing highlights [19]. However, there are also studies that conclude that reading a text that has been highlighted by expert readers improves recall of the highlighted text passage [13, 15]. This also implies that the quality of the highlighting plays an important role, as inappropriate highlighting can have negative effects on reading comprehension [20, 37]. The highlighting approach in contrast to using word clouds retains the context in which the word occurs and may provide more information at a glance. According to the guidelines on text highlighting techniques in [39], we chose to color the background of the 50 most relevant keywords in yellow. Figure 1B shows an extract of a highlighted text, which we used in our study.

3.3 Mind Map

Mind maps [9] are often used for organizing information by grouping important aspects into clusters and thereby have the ability to structure the information [43]. Structured information can be learned better and is processed more efficiently [4, 26]. We assume that due to the visualized connection, the gist of a text might be derivable and prime the reader for the text. The mind maps used in our studies are manually created, and we concentrate only on a few keywords and their relation to each other, to make the mind maps clear and usable. During initial trials we found about 15 keywords to be appropriate to represent our study texts whose length varied between 500 and 600 words. Some of them are grouped together to describe a single concept, e.g., the keywords “successful”, “learning” and “happiness” are part of the concept: “successful learning leads to happiness”. Figure 1C shows a summary of one of the texts used throughout our study.

3.4 Image Collage

Following the ancient proverbs “every picture tells a story” or “a picture says more than thousand words”, an image collage seems to be a suitable way to summarize text visually. Results of studies performed throughout the 1990s support this assumption as they have shown that depictive images aid the learning process [11]. Zhu et al. [44] presented a Text-to-Picture system capable of creating an Image Collage of short texts. The images are synthesized from some of the text’s most relevant keywords. Another method was introduced by Aramini et al. [3]: their system generates unsupervised and automatically visualization of shortened texts using images that can be found on the Web. We pursue a similar approach and search for suitable images corresponding to important keywords on creative common image portals like Flickr or Google Creative Commons Image Search. Not all of the 50 keywords can be well represented by an image. For creating the image collages for our user study, we analyzed the keywords regarding their metaphorical expression. We conclude that for each text eight pictures representing the main keywords can be chosen (see Fig. 1D).

4 Pre-study

To collect initial qualitative feedback on our general approach we conducted a pre-study in the form of focus groups.

4.1 Methodology and Procedure

We held two focus groups with four participants (3 females) each lasting about 90 min. All participants were university students with different backgrounds and between 19 and 24 years old (\(M=22.25, SD=1.67\)). The native tongue of everyone was German. We chose four texts with different text styles (fiction, news and science) and created four visualizations whereby the word cloud visualized a fictional text, highlighting was used for an abstract of a scientific paper, the mind map summarized a news article and the image collage a scientific article. Discussions were recorded in a written protocol, after which the participants’ comments were sorted into categories. Both visualization and text were in German and printed on paper. The main part of the focus group comprised discussions on the four visualizations regarding their usage and utility before, during and after reading the text. The four visualizations were discussed one after the other, and in the end, general design aspects of such visualizations were debated.

Results and Discussion. From the discussions and preliminary design explorations we derived ideas for using visualizations as pre-, during-, and after-reading support.

Pre-reading. Before diving into the actual text, participants found the visualizations helpful in general but mind map and image collage in particular. On the other hand, however, they were sometimes confused as some content in the visualizations did not make immediate sense to them. In some cases, the keywords were even found misleading. Furthermore, it was mentioned that visualizations could also help deciding whether a text is relevant to a certain topic.

During-Reading. Having visualizations accessible during reading, participants found it easier to keep an overview of the structure of the text. One participant stated that the mind map visualization helped to structure the text while another said the highlighting helped to pay attention to important parts of the text.

Post-reading. All participants found the visualizations to be very helpful in the post-reading phase, especially as memory aids when they needed to retain the texts’ contents. One participant had doubts about the usefulness of pictures as a memory aid after a long period and supposed textual cues would be more helpful. Another participant found post-reading support more important than pre-reading support.

In general, participants preferred getting a quick overview of a text. If a lot of time was required to look at a visualization to make sense of it, it was regarded as less beneficial. Participants said they had to spend the most time trying to figure out text contents from the word cloud, whereas the content of the image collage visualization was perceived quickest. During the study, no clear preference for one type of visualization could be observed. Especially the use of images as cues was controversial: some participants stated they found images most helpful, while others considered them least informative and preferred textual cues. Because no clear preferences for one of the visualization could be determined within the focus groups, we designed a user study for investigating all four visualizations. We set out to research the effect of such visualizations on the reader before, during and after reading. We start with the before-reading phase and present this user study as well as its results in the following sections.

5 User Study

With the goal of investigating the feasibility and appropriateness of our four text visualizations to prime readers for going through a text on electronic devices, we conducted a user study. As the Priming Effect is complex and cannot be regarded as a unitary effect [35] we focus on different aspects to approximate it:

  1. 1.

    The first impression the reader gets about the text after seeing the visualization but before reading it.

  2. 2.

    The effects on reading comprehension and time spent reading.

  3. 3.

    The subjective utility of visualizations and how they help readers gain an overview of a text and its content.

5.1 Apparatus

As reading material, we chose five standardized texts from the Institute for German as a Foreign Language (TestDaF). Since we expected participants to be native German speakers, we selected texts in German dealing with issues of general interest (e.g., “Earthquakes” or “The effect of nuts on health”). Text lengths ranged between 500 and 600 words. Each text came with ten multiple choice questions, each with three possible answers for assessing reading comprehension. For each text we extracted the 50 most relevant keywords: each text was preprocessed with TreeTagger [36]. From its output, we only considered nouns, verbs, and adjectives as well as removed frequent stopwords. For tf-idf calculations, we used the gensim libraryFootnote 1 and used a corpus consisting of 500 articles, which we took from SSOAR Footnote 2, a freely accessible paper platform for social sciences. For creating the word clouds we used JavaScript. Highlighting visualizations were manually created based on the words extracted from our system. The image collage was also manually created and used the eight most important keywords, for which we searched images through Google Images. The mind map was manually generated, too, but was oriented on the text’s structure and keywords. Keyword extraction, as well as the creation of the visualizations, was done independently from the comprehension tests in order to not induce any researchers’ bias. Figure 1A–D shows visualizations used for the text “Learning-Pleasure or Burden”. Since our investigation focuses on reading on electronic devices, we conducted the study on desktop PCs and therefore created a comprehensive online survey using Limesurvey Footnote 3. Hence, multiple participants could take part in the study at the same time under our supervision.

5.2 Methodology

Data collection took place from August to September of 2016 at a university and a research institute where multiple participants could be processed simultaneously. We followed a within-subject design approach with the texts and the visualizations being the independent variables in five conditions: the four different visualizations, plus one text without visualization as a baseline condition. As dependent measures, we collected Likert-style ratings, comprehension scores, measured reading time, and free-form text comments.

Table 1. Structure of the survey. In the baseline condition (no visualization) only the steps ‘Text’ and ‘Comprehension questions’ were utilized in section B.
Full size table

5.3 Procedure

At both locations, the study took place at a computer lab under controlled conditions without distractions on 13 in. displays showing the texts with a 12px font size. All participants followed the following procedure: we explained the purpose of the study and asked each participant to sign a consent form. Then, we handed out a paper with the URL to the online survey, which they could fill in at one of the workstations in the lab. Participants were randomly but equally assigned to five different surveys. Each survey employed a different order of conditions as well as different allocations between the condition and underlying text. Participants were assigned to the surveys following a counterbalanced measures design via Latin Square. The surveys were divided into steps in a series of self-directed webpages (see Table 1). It started with an introduction and questions about demographic information and reading behavior. Afterwards, the visualization of the text was shown for one minute. All visualizations except for highlighting were fully visible. For highlighting participants could freely scroll the text. Participants were allowed to progress with the study before one minute was over by clicking on “next”. A question about the content of the text followed before the actual text was presented to participants. There was no time-limit for reading the text. Then, questions about the visualization were asked, before ten comprehension questions had to be answered. This step B (see Table 1) was repeated for each visualization and the baseline condition. In the baseline condition, participants had only to read the text and to answer the comprehension questions. The study took about 45 min for each participant.

5.4 Participants

We recruited 50 participants (15 females) with their age ranging from 18 to 60 (\(M=27.7, SD=6.19\)). 32 were university students from different fields of study, while 10 were researchers and the remaining seven worked as software developers or secretaries. 45 participants stated German to be their mother tongue, and the others spoke German fluently. 21 participants were considered to be frequent readers (more than seven hours reading time per week), 12 to be casual readers (between four and six hours reading time per week) and 17 to be scarce readers (less than three hours reading time per week).

6 Results

In the following, we present our results from the subjective assessments, pre-impression analysis, comprehension tests, reading time, and free commenting.

6.1 Quantitative Measures

User Ratings. For the following analysis, we applied Friedman tests followed by Wilcoxon signed-rank tests with a Bonferroni correction applied, which resulted in a significance level of \(p<0.008\). Figure 2 contains an overview of all subjective ratings with regard to each visualization and the four characteristics: Comprehension, Gist, Fit and Structure as shown in Table 1.

Fig. 2.
figure 2

Subjective user ratings (mean and standard errors) for each visualization type with regard to their utility for supporting text comprehension, understanding the gist of a text, its fit to reflect the text’s content and its adequacy to reflect the text’s structure.

Full size image

Looking at the subjective user feedback regarding the utility of the visualizations, there was a statistically significant difference in perceived comprehension with regard to visualization type, \(\chi ^2(3) = 33.725\), \(p<0.001\). Median perceived comprehension ratings for word cloud, mind map, highlighting, and image collage were 2 (1 to 3), 4 (2.75 to 4.25), 3.5 (2 to 4), and 2 (1 to 3), respectively. There was a statistically significant difference in perceived comprehension in favor of mind map over word cloud (\(Z = -4.36\), \(p<0.001\)) and over image collage (\(Z = -4.022\), \(p<0.001\)). Also, highlighting was rated significantly higher than word cloud (\(Z = -2.636\), \(p=0.008\)) and image collage (\(Z = -3.344\), \(p<0.001\)).

As for perceived text content overview—i.e., getting the gist of a text—there was a statistically significant difference in how much of the different visualization types conveyed, \(\chi ^2(3) = 38.173\), \(p<0.001\). Median perceived content overview ratings for word cloud, mind map, highlighting, and image collage were 4 (2 to 4), 5 (4 to 5), 4 (3 to 4), and 2 (1 to 4), respectively. Post-hoc tests showed a statistically significant difference in how well the visualizations provided a content overview with mind map being rated best over word cloud (\(Z = -4.336\), \(p<0.001\)), image collage (\(Z = -4.534\), \(p<0.001\)), and highlighting (\(Z = -2.806\), \(p=0.005\)). Highlighting was better received than image collage (\(Z = -3.998\), \(p<0.001\)). No statistically significant difference was found between highlighting and word cloud (\(Z = -1.496\), \(p=0.135\)) and between word cloud and image collage (\(Z = -1.678\), \(p=0.093\)).

There was a statistically significant difference in how comprehensive the corresponding visualization fit the text content, \(\chi ^2(3) = 36.491\), \(p<0.001\). Median perceived ratings for word cloud, mind map, highlighting, and image collage were 2 (1 to 3.25), 4 (2 to 4), 3 (2 to 4), and 2 (1 to 2), respectively. There were statistically significant differences with mind map being rated better than word cloud (\(Z = -3.822\), \(p<0.001\)) and image collage (\(Z = -4.627\), \(p<0.001\)). Also, highlighting was better perceived to comprehensively reflect the content than image collage (\(Z = -3.719\), \(p<0.001\)). No statistically significant difference was found between highlighting and word cloud (\(Z = -1.984\), \(p=0.47\)), image collage and word cloud (\(Z = -1.921\), \(p=0.055\)) or between highlighting and mind map (\(Z = -2.169\), \(p=0.03\)).

Finally, results yielded a statistically significant difference in how well visualization provided an overview of the text structure, \(\chi ^2(3) = 48.850\), \(p<0.001\). Median perceived ratings for word cloud, mind map, highlighting, and image collage were 2 (1 to 4), 4 (3 to 5), 4 (2 to 4), and 2 (1 to 2), respectively. Post-hoc tests showed a statistically significant difference in structural overview ratings with mind map be being rated better than word cloud (\(Z = -4.599\), \(p<0.001\)) and image collage (\(Z = -5.224\), \(p<0.001\)). Highlighting was better perceived than word cloud (\(Z = -2.940\), \(p=0.003\)) and image collage (\(Z = -4.670\), \(p<0.001\)). No statistically significant difference was found between word cloud and image collage (\(Z = -1.965\), \(p=0.49\)) or between highlighting and mind map (\(Z = -2.136\), \(p=0.033\)).

Pre-impressions. After being shown each visualization and before actually being presented the full text, participants were asked about their pre-impression, i.e. what they thought the text’s content was about. We had three independent researchers assess each comment according to how well they fit the actual text content on a school grading scale from 1 (being best) to 6 (being mislead). Therefore, we instructed the researchers to assign grade 4 when the rough topic was identified, 5 when the topic was incorrectly identified but was at least similar, and 6 when the participant’s assumption did not match the content of the text at all. Grades from 1 to 3 depended on the level of detail. We then calculated a mean inter-annotator agreement score for each pre-impression statement and performed a Friedman test on the resulting final grade. There was a statistically significant difference in calculated grades based on the quality of participants’ pre-impressions, \(\chi ^2(3) = 47.730\), \(p<0.001\). Median perceived ratings for word cloud, mind map, highlighting, and image collage were 4 (3 to 4.25), 3 (2 to 4), 4 (3 to 4), and 4 (4 to 5), respectively. There were statistically significant differences with mind map leading to most accurate pre-assessments compared to tag cloud (\(Z = -3.820\), \(p<0.001\)), highlighting (\(Z = -3.987\), \(p<0.001\)), and image collage (\(Z = -5.393\), \(p<0.001\)). Also, tag cloud performed better than image collage (\(Z = -3.053\), \(p=0.002\)), so did highlighting (\(Z = -4.046\), \(p<0.001\)). Highlighting and tag cloud seemed to perform similarly resulting in a non-significant difference (\(Z = -0.767\), \(p=0.443\)).

Comprehension Scores and Time Spent Reading. For comprehension scores, a one-way repeated measures ANOVA with a Greenhouse-Geisser correction yielded no statistical significance between conditions (\(F(3.314, 162.402) = 1.086\), \(p=0.36\)). We looked at how much time it took participants to read the prospective texts followed by studying the visualizations for 60 seconds. Most participants took full advantage of this time interval. Hence, there was no statistically difference in visualization viewing time (\(F(1.158, 56.731) = 0.296\), \(p=0.622\)). With regard to time spent reading the full text, there was a significant difference based on which visualizations participants studied beforehand (\(F(3.335, 163.435) = 3.087\), \(p=0.024\)). Post hoc tests using the Bonferroni correction revealed that readers spent less time reading after a highlighting visualization compared to viewing an image collage (\(Z = -3.384\), \(p=0.001\)). On average, readers who viewed a highlighted version of the text were 24.9 seconds (i.e., 12.1%) faster than when viewing an image collage (compared to word cloud: 10.4%, mind map: 5.5%, and without visualization: 9.2%) No other pairwise comparisons yielded any statistically significant results, however there is a tendency of highlighting to help reduce reading time as compared to word clouds (\(Z = -2.736\), \(p=0.006\)) and reading without any prior visualizations (\(Z = -2.667\), \(p=0.008\)).

Fig. 3.
figure 3

Participants’ ratings of different visualization types regarding appeal and helpfulness.

Full size image

6.2 Qualitative Measures

Participants were asked which visualization they found most appealing and most helpful (see Fig. 3). They were encouraged to provide explanations of their assessment, which we analyzed by dividing them into single statements and assigning them to respective visualizations. Altogether, we collected 32 statements regarding appeal and 52 statements regarding helpfulness. In the following, answers given by more than one participants are listed and the number of occurrences is shown in brackets after each provided statement: the image collage was described as a simple and clear design, but at the same time with a lack of expressivity (N = 3). It is clearly arranged (N = 3), arouses interest to read the text (N = 2), is good to remember things (N = 2) and helps to formulate a mental image of the content (N = 2). On the other hand, participants pointed out that “Pictures are open to interpretation, therefore, they might mislead the reader.” (N = 2). Highlighting provides the structure of the text (N = 5) and shows the words in their context (N = 4). However, two subjects were overstrained by the volume of text. Comments for the mind map were predominantly positive. It was stated as clearly and logically structured (N = 9). It summarized the essential information of the text (N = 11), shows connection between the information (N = 10), allows recognizing the structure of the text (N=9) and contributes to the understanding (N = 4). Participants appreciated the word cloud for providing the most important words (N = 4) in a pretty way (N = 3). But the word cloud was also considered as too overloaded (N = 2). In general, there was a perceived benefit of looking at text visualizations before reading a text. This is supported by the fact that nobody answered the question “Which visualization do you find most helpful?” with none.

7 Discussion

The initial focus group gave us insights into the applicability of visualizations before, during, and after the actual reading process. Due to the feedback, we set out to assess the feasibility of using text visualizations as primers to improve text reading. Therefore, we conducted a user study examining the utility of four different visualizations types.

We could not confirm an objective effect difference on comprehension between those visualizations. When designing studies around reading it is crucial to not only pay attention to the nature of the texts used, but also to the difficulties of measuring text comprehension. We opted for a solution in which the texts came with pre-defined comprehension tests as they were commonly used in current language proficiency assessments. Hence, they often required not only literal translation, but also transfer skills. When reviewing the test results, we noticed that many participants had difficulties with these type of questions resulting in a rather homogeneous distribution of mean comprehension scores, which is why we also analyzed the subjectively perceived effects that visualizations had on readers. Here, we found effects on subjective perception of using visualizations before reading with regard to their utility to convey the text’s content and structure. The mind map was rated most helpful in providing an overview and getting the gist of a text. Its nature allows communicating content and structural information more comprehensively since the mind map’s visual design often reflects semantic and structural relationships in the text.

Also, highlighting lead to better subjective text comprehension. By studying highlighted keywords of a text before reading it was shown in our study to give readers an impression of the text’s gist. We also found that it reduced the actual reading time by up to 12.1%. This could be due to the readers already being exposed to the entire text beforehand, where participants were able to read ahead. However, some of the participants claimed that due to the time limit of 60 seconds they had problems to capture all of the highlighted keywords, which might be an argument against the premature detailed reading.

By objectively assessing the quality of participants’ presumed text summaries prior to reading, we find that visualizations can give readers a good first impression of the content. Mind maps—due to being rich in structure and content information—helped participants’ most in grasping the text content, while highlighting and word cloud performed better than image collage. These findings show the usefulness of visualizations, such as mind maps, to allow readers to skim and possibly disregard a text quickly. This skill is especially useful in situations or jobs, where people need to process a lot of text.

We were further able to relate effects of inappropriate highlighting to participants’ feedback stating that visualizations could be misleading. One participant pointed out that highlighting a verb without its negation might be interpreted completely different. The benefit of highlighting is strongly dependent on the quality of the underlying keyword extraction. As Silvers and Kreiner [37] showed that inappropriate highlighting could have negative effects on reading comprehension, merely using tf-idf scores might be too simplistic. More sophisticated topic models, such as Latent Dirichlet Allocation (LDA), need to be exerted in order to extract meaningful keywords and therefore convey text context more accurately to support comprehension. Image collages also bear the risks to prime the user in different directions. This is caused by the rather ambiguous nature of images since they often trigger different associations, which is challenging to design for deliberately.

During the pre-study, we identified three strands of application scenarios, namely using visualizations as primes before reading, as an aid to keep an overview during, and as a summary support for after reading. A combination of those could have profound effects on readers’ ability to quickly get the gist of a text in order to decide on its relevancy. Then during reading contextual graphics are provided to aid readers’ sense of text position and sequence of argumentation. And afterwards, having a summary highlighting the text essence to be committed to long-term memory. Specific visualizations during each of these stages can differ due to their utility, which is why we investigated the first stage with all four visualization types mentioned. Further studies will need to be conducted to get insights into their specific design, most effective content generation, and application cases. For now, we have been focusing on the pre-reading phase and the use of such visualizations to prime readers. Based on our quantitative and qualitative findings we derive a set of implications in the following and introduce our vision for an automated system that takes text cues from a user’s reading device and provides effective visualizations tailored to the reader to support the reading experience.

8 A System to Support Readers Through Visualizations

In our study of visualizations, we found that prior viewing impacted the readers’ ability to make out the gist of a text and in the case of highlighting reduced the actual reading time. Mind maps, on the other hand, were best fit to convey the structure and the essential content of a text.

In the following, we apply our findings into a vision of an automated system capable of providing such visualizations on the fly as users deal with their daily readings. Thereby, not only the text at hand needs to be considered, but also the reading goal. Visualizations, for example, can help readers disregard or engage with a text in more detail based on their pre-assessment. Further, as discussed by our focus groups, visualizations can serve different purposes based on whether they are being used before, during, or after reading. A system that takes a text as an input and creates visualizations on the fly can tailor to those requirements. Figure 4 shows an overview of the key components of such a system, which takes text from the reader’s device as cues to generate visualizations while considering the user’s reading goal and characteristics (such as background knowledge and reading habits). From our results, we derived the following important aspects that we propose to be considered while creating such a system.

Fig. 4.
figure 4

System pipeline for supporting in-situ text visualizations: a central instance collects available texts, processes them, links them to additional content, such as relevant images, and creates visualizations taking into account readers’ preferences and the devices’ attributes on which these visualizations are being rendered.

Full size image

Extraction. An automated system for reader support through visualizations needs to comprise a representative corpus to ensure a reliable keyword extraction. The extraction itself and its quality are vital for the utility of the system, as we have learned from the participants’ feedback who were not always able to connect the topics presented in some visualizations, especially in image collages, which mislead some readers. A reliable keyword extraction then informs the system which images to look for in order to enhance visualizations or build the corresponding image collages.

Visual Clues. Providing visual clues about the semantic link between the keywords is crucial since we have learned that keywords on their own are not meaningful enough. Such links can be of structural nature, as the mind maps showed to be more effective when conveying text overview. Such links—also to externally related sources—can help readers understand content since providing context increases text comprehension [8].

Image Selection. Images are appealing and can be useful to motivate readers to read a text, but bare little insights about the gist of a text and can even be misleading. For an automated system, there are two important aspects:

(1) filtering out meaningful keywords that describe a text in its essence, and (2) a comprehensive image database that contains a well-indexed pool of images to avoid ambiguities. In the study presented, we focused on the applicability of image collages as primes, hence we opted for manual image selection, but sophisticated tagging and visual processing algorithms might automate this process in the future.

Address Information Overload. For pre-assessments of text, highlighting of most relevant text aspects is vital to convey the gist of a text effectively. Avoiding clutter is crucial since otherwise the visualization overloads the reader and loses its merit alongside the text. In our study, it became clear that mind maps conveyed a stronger structural overview as compared to word clouds, which were more likely to be perceived as cluttered. The hierarchical structure of mind maps helps to highlight important concepts where details can be pushed further down the branches.

Text Structure. Conveying a text’s structure helps readers orient themselves in the text and allows them to follow the string of argumentation. The mind maps used during our study conveyed that structure, which leads to increased overview of structure and content. The simple structure is given by text flow, but using semantics and language processing, a chain of argumentation could be extracted from text and displayed accordingly.

User Characteristics. People are inherently different in reading aptitude, their interests, and background knowledge. A system that learns about user preferences regarding preferred topics, language, and reading device, could adapt to these needs. Such preferences could be collected by explicit user settings or implicitly derived by tracking and analyzing reading content and reading behavior over time. Also, real-time assessments of user states regarding visual fatigue and engagement can help adjusting visualizations to match users’ cognitive capacities better [27], e.g., creating more complex visualizations when readers are engaged vs. simplifying content when they are fatigued. Based on such internal user states, a dynamic reading interface could further be instructed by the system to guide the reader quicker through the essential parts of a text [16].

Taking these aspects into account, we see great potential for visualizations as text primes. Future work includes the integration of an automated system as depicted in Fig. 4, so that readers can get in-situ cognitive support as they deal with their daily readings on electronic devices. By receiving visualizations that break down the gist of a text into small, digestible chunks. Such visualizations can guide readers through the string of argumentation and support text comprehension.

9 Conclusions

In this paper, we focused on the use of visualizations as primers to support reading. In a comprehensive user study with 50 participate we investigated in what way four types of text visualizations were useful to convey text content and structure before reading. Comparing word cloud, highlighting, mind map and image collage we found that mind maps were rated best to support comprehension and structure of a text. Highlighting, on the other hand, showed that readers who study keywords in their context later require less time spent reading.

We conclude with a proposed design of a system that receives a text and provides context-specific visualizations. Context includes text cues produced by keyword extraction, the combination of different sources (e.g., image databases), and taking into account specific goals and characteristics of the reader. This could allow visualizations not only to serve as text summaries but further convey text structure and provide enhancements through images. Our study showed that readers were able to pre-assess the content of a text based on visualizations—foremost through mind maps, which allowed them to absorb the gist of a text quickly. With the growing demands of our information society to keep up with daily readings, visualizations can help readers making quick and informed decisions about the utility and content of a text prior to reading.