Introduction

Human movement is an integral part of the perceptual-motor system. People constantly use their arms, hands, and fingers in simple and complex activities, as for example, reaching objects, pointing at visual targets, typing on a keyboard, or moving a computer mouse. In all these activities, movements can be very complex and their study, in terms of kinematics and/or simple trajectories, can provide a simple and valid basis to uncover cognitive and behavioral processes underlying the observable structure of the data. Recently, many authors have focused on the analysis of human movements measured by means of a computer mouse, an approach called mouse tracking methodology (e.g., see: Freeman & Ambady, 2010; Hehman, Stolier, & Freeman, 2015). The basic idea is to consider the collected mouse movement trajectories as a continuous source of real-time information on the internal cognitive processes activated during a particular experimental task. For instance, in a dichotomous categorization task where two alternative choices are presented on a computer display (e.g., target vs. distractor), mouse trajectories can reveal, according to the experimental manipulation, the eventual competitive attraction that one of the two choices (e.g., distractor) acts on the correct target category (Spivey and Dale 2006). This simple idea has recently been applied in several research fields, including social categorization (Dale et al. 2007), moral decision-making (Koop 2013), language comprehension (Morett and Macwhinney 2013), numbers representation (Faulkenberry 2014), and cross-modal processing (Coco and Duran 2016).

Several strategies are available to analyze mouse tracking data both from a model-dependent framework (e.g., Friedman, Brown, & Finkbeiner, 2013; Mottet & Bootsma, 1999; Plamondon, 1995) and a model-independent framework (e.g., Freeman & Ambady, 1988; Meyer, Abrams, Kornblum, Wright, & Keith Smith, 2010). Although the model-dependent approaches have a long tradition in cognitive psychology (e.g., Wiener’s diffusion models, Rayleigh–Duffing model, minimum-jerk kinematics), the data-driven model-independent approaches are still appealing in terms of simplicity and feasibility as a considerable number of papers have also shown recently (Freeman 2016). In this paper, we will describe a novel mathematical procedure, called EMOT (Entropic MOuse Tracker), to analyze mouse-movement trajectories from a data-driven perspective. In particular, EMOT extends the main findings of the descriptive geometric approach (Freeman and Ambady 2010) in analyzing the basic features of the mouse trajectories (i.e., directions and amplitudes). However, unlike the latter approach, our proposal provides a more robust modelization of the trajectories in case of noisy and irregular movement paths and, in addition, it models a previously ignored component of the hand movement, motor pauses, which can reflect important aspects of the reaching process, for instance decisional uncertainty in classification tasks (Plamondon 1995; Johnson et al. 2012; Glaholt and Reingold 2011).

The reminder of this article is organized as follows. In the “Analyzing movement trajectories: a brief overview”, we provide a comprehensive overview of the perspectives used in analyzing mouse trajectories. In the “An entropic approach to mouse-tracking data”, we present our proposal together with its main characteristics. The “Two empirical applications” describes two case studies showing the application of the new proposal to two psychological datasets. The “Discussion” provides a discussion of the results of this study with comments and suggestions for future works whereas the “Conclusions” ends the article.

Analyzing movement trajectories: a brief overview

To formally represent goal-directed movement trajectories, two main approaches can be recognized in the model-dependent framework. In the first one, movement paths are evaluated by fitting probabilistic models on the collected data (e.g., Wiener’s diffusion models, dynamic Markov chains) with the purpose of discovering important movement features, such as initiation process, decision phases, changes of direction (Friedman et al. 2013). Although this approach is quite successful in modeling mouse trajectories, in some circumstances (e.g., when the model assumptions cannot be completely satisfied) it might lead to problems of data representation (Voss and Voss 2008; White et al. 2010). In the second perspective, movement features like speed, acceleration, stability, and frictions are formally deducted using mathematical models (e.g., the Rayleigh–Duffing model - Mottet & Bootsma, 1999) and kinematic principles (e.g., minimum jerk, minimum energy - Engelbrecht, 1995; Flash & Henis, 1991; Plamondon, 2001). This approach has been widely adopted to study well-constrained class of movements usually recorded in simple tasks, as for instance smooth target-directed movements, planar movements, reaching with obstacles. However, to what extent it can account for highly structured and noisy movement paths is still under debate (Shadmehr 2005). Unlike the model-dependent approaches, the model-independent ones make use of a data-driven perspective to detect process features related to hand movements. In particular, they can be based either on some computational methods for movement decomposition such as, initiation time, pause time, and verification time (Walker et al. 1993; Meyer et al. 1988) or on basic descriptive geometric method to detect significant movement features, such as pointwise distances (Freeman and Ambady 2010; Hehman et al. 2015). Because our new proposal is mainly associated with this latter representation, in the following section we will briefly review the geometric approach.

Descriptive geometric approach

The descriptive geometric approach (DGA) gathers different qualitative analytic tendencies that have been commonly used to uncover cognitive features associated to the observable motor behaviors of the hand during mouse-tracking experiments involving choice, categorization, and decision-making tasks (Freeman and Ambady 2010; Song and Nakayama 2009). It has been widely applied in two choice-categorization tasks where participants are asked to select, by means of a computer mouse, the appropriate category of a stimulus against a competing alternative (Fig. 1). The underlying hypothesis here is that the stream of motor information recorded by the computer mouse might be considered as a window on the ongoing cognitive competitive dynamics involved during the task (e.g., the more two categories are similar, the more they are difficult to discriminate, the more the curvature is toward the incorrect category).

Fig. 1
figure 1

Conceptual diagram of DGA in a two-choice categorization task. (a) Categorization schema with target C2, competitive category C1, and an hypothetical mouse path in green. (b) Cartesian mouse trajectories in two-dimensional plane (the bisector of the I quadrant indicates the ideal movement). (c) Example of two descriptive measures of the movement path: gray indicates the area under the curve (AUC) whereas the red straight line indicates the maximum deviation/distance (MD)

DGA basically implements a simple data-analysis procedure. First, the raw computer-mouse trajectories are pre-processed by space-rescaling, time-normalization, and eventually averaging. Next, the ensuing trajectories are carved up by using various measures, such as maximum deviation (MD), area under the curve (AUC), x-y flips, sample entropy, and bimodal coefficient, which are all supposed to be good indicators of the ongoing dynamics underlying the cognitive tasks (see Fig. 1). These measures are finally used in post hoc analyses, like spatial attraction analysis, distributional analysis, and spatial disorder analysis (Hehman et al. 2015; Freeman and Ambady 2010).

However, despite its simplicity, DGA can suffer from a number of potentially relevant drawbacks. First, the time-normalization procedure can sometimes disregard important noise which may be instead signal (e.g., see: Lucero & Koenig, 2000; Tang & Müller, 2008). Second, in some circumstances, averaging multiple movement profiles may lead to inappropriate results, especially when empirical paths of movement show different amplitudes, phase variations, and curvatures (Ramsay 2006). Consequently, it is hard to choose a proper averaging operator (e.g., arithmetic, geometric, block-based) that preserves the profile information (Brown and Heathcote 2003). Third, due to the possible problem of multiple-maxima in the raw movement profile, the spatial attraction analysis may be biased. Indeed, as reaching movements are noisy, the recorded movement profiles may not completely show clear patterns with just one inflection point (this is the case of complex sinusoidal and quasi-cyclic trajectories). As a result, spatial measures like MD, AUC, and x/y flips may not be sensitive enough to recognize relevant subcomponents of the movement. Finally, the spatial measures cannot capture features or information associated with the presence of motor pauses during the motor execution. However, these pause patterns can potentially reveal important characteristics of the dynamic of a cognitive process such as, for example, decisional conflicts or uncertainty in categorization or choice tasks (Johnson et al. 2012; Glaholt and Reingold 2011; Plamondon and Alimi 1997).

In the next section, we will present an alternative mathematical method, EMOT, for movement trajectories that can overcome many of the drawbacks of DGA by providing a good compromise between model flexibility and model simplicity. In particular, EMOT extends many of the major conclusions from DGA (e.g., hand movements can provide real-time indicators of cognitive processing, dynamic competition during categorization can be capitalized by abrupt shift of movement profile) by using a model-based representation for the computer-mouse trajectories. In addition, this method will also provide a valuable way to detect other movement events (e.g., motor pauses) that may potentially reveal decisional uncertainty and/or conflicts during the cognitive task.

An entropic approach to mouse-tracking data

Generally, a computer-mouse trajectory can be characterized in terms of fast movements, which correspond to rapid motor executions in the movement space (see Fig. 2a, points in green), and motor pauses, occurring when the speed of the mouse cursor decreases and eventually breaks in a given region of the movement space (see Fig. 2a, points in the red ellipse. For further readings see: Hwang, Keates, Langdon, & Clarkson, 1988; Meyer et al., 2005).Footnote 1 By adopting an information-theoretic perspective, EMOT aims to quantify the decision process, which is codified by spatial components of computer-mouse trajectories, with a set of entropy measures within a single-trial approach. In particular, the noisy and dynamic x-y trajectory is initially mapped into a set of polar objects (distances and angles). Next, a linear histogram model of angles of movements is built, which stores the most important spatial features of the computer-mouse trajectory such as location, directions, and amplitudes. Finally, the histogram model is used to determine a quantification of spatial events involved in the original movement path (e.g., competition between target and competing cues present on the screen) in terms of fast movements and motor pauses. The final step is realized according to a tailor-made entropy decomposition of the histogram model. In this way, similarly to DGA, our method will provide information concerning the spatial properties of a trajectory (i.e., location, direction, amplitude) with regards to both the target and competing cues. However, unlike DGA, EMOT supplies a model for the noisy mouse-movements and, in addition, a way to represent motor pauses that are usually neglected in the DGA approach. In fact, motor pauses may reveal important aspects of the dynamic cognitive processes such as, for example, decisional conflicts in categorization and/or decisional uncertainty associated with the participants’ behaviors in the experimental task (Johnson et al. 2012; Glaholt and Reingold 2011; Plamondon and Alimi 1997).

Fig. 2
figure 2

Conceptual diagram of EMOT modeling: from mouse trajectories to histogram of spatial motion information (points of motor pause in the red ellipse, points of fast movement in green). (ac) Hypothetical histograms of spatial motion information: (a) Pattern with attraction toward C1; (b) Pattern with equally distributed attraction; (c) Pattern with no attraction toward C1. Note that C1 and C2 refer to distractor and target, respectively

Data representation and modeling

Let \(\mathbf {p} = (\mathbf {x}, \mathbf {y}) \in \mathbb {R}^{n} \times \mathbb {R}^{n}\) be an array of Cartesian coordinates associated to the streaming of x-y data of the computer mouse movements with n being the recorded number of movements points for a given participant in a specific trial. The modeling step consists of transforming the mouse movement trajectory into a spatial motion model, while retaining as much as possible of the properties of the original trajectory (i.e., location, direction, amplitude) present in the original movement space. Inspired by some previous works in the field of action recognition (Gowayyed et al. 2013; Kapsouras and Nikolaidis 2014; Chen et al. 2015) and topic analysis (Wang et al. 2013), we used the linear histogram of movement angles as a simple trajectory model (Shadmehr and Mussa-Ivaldi 2012; Calcagnì and Lombardi 2014). The use of a histogram model simplifies the study of trajectories and spatial features in a proper low-dimensional movement space. This facilitates, for instance, the emergence of relevant movement properties like the linear spatial attraction of the distractor on the target cue. More technically, let p =(r,𝜃) be the transformed vector of p, where \(\mathbf {r} = \sqrt {\mathbf {x}^{2} + \mathbf {y}^{2}}\) and 𝜃=atan2{y,x}. In r the power is element-wise whereas, for the sake of representation, the function atan2{.} is implemented using a special clockwise version which works in the subspace [0,3.14] (half quadrant). Next, the vector of angles \(\boldsymbol {\theta } \in \mathbb R_{+}^{n}\) is modeled by the histogram \(\mathcal {H}_{\theta } = (\boldsymbol {\eta }, \boldsymbol {\omega }) = \{(\eta _{1}, \omega _{1}),...,(\eta _{t}, \omega _{t}),...,(\eta _{T}, \omega _{T}) \}\) where η t is the t th bin (t=1,…,T), ω t represents the frequency of occurrence of movement points in η t , and T is the natural histogram parameter (Fisher 1995). The search for the best number of bins T is an important point in the construction of the model \(\mathcal {H}_{\theta }\). As is known, several rational criteria (e.g., Sturges’ formula, Doane’s formula, Freedman–Diaconis’ rule) as well as automated procedures (e.g., see: Shimazaki & Shinomoto, 2007) can be employed to get appropriate estimates of T. In our context, this search is automated using a dedicated trial-by-trial sensitivity analysis (Birgé and Rozenholc 2006).Footnote 2

Figure 2 shows a graphical representation of the EMOT modeling step. In particular, panel (A) represents the prototypical case in which the competing cue C1 produces attraction. In this example, the support of the histogram spans (in angles) from C1 to C2 although the majority of frequencies is still located around C1. On the contrary, panel (C) reflects a situation in which mouse movements are largely located toward the final selected cue C2 while showing fewer directions toward C1 even though they span along the interval ranging from C1 to C2. This situation might occur when fast and hasty mouse movements do not relate with a real attraction for the competing cue C1. Finally, Panel (B) codifies a situation where both C1 and C2 cues produce a strong attraction on mouse movements with the consequence of an higher recorded uncertainty.

Extracting motion features

To quantify fast movements and motor pauses from movement trajectories, we first transformed the linear histogram by means of an operator ϕ(η,ω), which returns for each distinct bin η t the corresponding cumulative frequency \({\sum }_{s \leq t}\omega _{s}\) smoothed according to a standard interpolation algorithm (e.g., see: Eilers & Goeman, 2004). Next, the cumulative residual entropy of ϕ(η,ω) was computed as follows:

$$ \psi = [\phi(\boldsymbol\eta, \boldsymbol\omega)-\mathbf{1}] \log[\mathbf{1} - \phi(\boldsymbol\eta, \boldsymbol\omega)]^{T}\mathbf{1} $$
(1)

where 1 is a T×1 vector of all ones. The quantity ψ is a modified version of the traditional Shannon entropy and has been firstly introduced by Rao et al. (2004). It is defined as a measure of the average amount of information and therefore is related with the degree of complexity in a given system (for further details see: Di Crescenzo & Longobardi, 2009; F. Wang & Vemuri, 2007).Footnote 3 In our context, ψ is an overall measure of the total spatial motion information captured by the movement model ϕ(η,ω), which represents the amount of information reflected in the movement trajectory. In order to account for the sub-components of the trajectory, Eq. 1 is additively decomposed, as follows:

$$ \begin{array}{ll} \psi &= \xi + \zeta \equiv [\phi(\boldsymbol\eta, \boldsymbol\tau)-\mathbf{1}] \log[\mathbf{1} - \phi(\boldsymbol\eta, \boldsymbol\tau)]^{T}\mathbf{1} \\&{\kern10pt}+ [\phi(\boldsymbol\eta, \boldsymbol{\upsilon})-\mathbf{1}] \log[\mathbf{1} - \phi(\boldsymbol\eta, \boldsymbol{\upsilon})]^{T}\mathbf{1} \end{array} $$
(2)

where ξ and ζ are the unknown residual cumulative entropies associated with the fast movement and the motor pauses, respectively. Similarly, τ and υ are T×1 vectors of unknown weights of the models of fast movements and motor pauses, respectively. Furthermore, by letting ζ = ζ 1 + ζ 2 to include the information related to competing and correct cues, Eq. 2 can be additionally decomposed as:

$$ \begin{array}{ll} \psi &= \xi + \zeta_{1} + \zeta_{2} \\ & \equiv [\phi(\boldsymbol\eta, \boldsymbol\tau)-\mathbf{1}] \log[\mathbf{1} - \phi(\boldsymbol\eta, \boldsymbol\tau)]^{T}\mathbf{1} + \\ & {\kern10pt}+ [\phi(\boldsymbol\eta, \boldsymbol{\upsilon}_{1})-\mathbf{1}] \log[\mathbf{1} - \phi(\boldsymbol\eta, \boldsymbol{\upsilon}_{1})]^{T}\mathbf{1}\\& {\kern10pt}+ [\phi(\boldsymbol\eta, \boldsymbol{\upsilon}_{2})-\mathbf{1}] \log[\mathbf{1} - \phi(\boldsymbol\eta, \boldsymbol{\upsilon}_{2})]^{T}\mathbf{1} \end{array} $$
(3)

where ζ 1 and ζ 2 are the unknown cumulative residual entropies of motor pauses occurred in the direction of the competing cue and the target (see Fig. 2a), whereas υ 1 and υ 2 are T×1 vectors of unknown weights of the corresponding components. The decomposition of the quantity ψ allows for the identification of the movement components and their separate analysis. The overall estimation problem boils down in determining the values of the unknown quantities ξ, ζ 1, and ζ 2 that satisfy the equality (3), given the observed data 𝜃.

Estimating ξ, ζ 1, ζ 2

In line with the descriptive nature of our proposal and in order to retain the single-trial approach upon which EMOT is based, we estimated ξ, ζ 1, ζ 2 by reformulating the decomposition as a non-linear optimization problem (NLP) within the minimum Kullback–Leibler framework (Ciavolino and Calcagnì 2015; Good 1963), a dual version of the well-known Jayne’s maximum entropy principle (Kapur and Kesavan 1992). However, in order to apply this procedure, we first need to define a set of external feasible information about ξ, ζ 1, and ζ 2 that will constrain the class of possible solutions of our decomposition problem.

As the only information available is contained in the vector of angles of movements, we can use 𝜃 to derive some proxies that can be used in solving our problem. In particular, let \(\boldsymbol {\theta }_{0} = \mathcal {F}_{0}(\boldsymbol \theta )\) be the zeroth-frequency moment of 𝜃 where 𝜃 0 contains only distinct element of 𝜃 with no ties whereas \(\mathcal {F}_{0}\) is a standard algorithm for this purpose (Coppersmith and Kumar 2004).Footnote 4 Like for \(\mathcal {H}_{\theta }\), we can also define the histogram for 𝜃 0. More specifically, let \(\mathcal {H}_{\theta _{0}} = (\boldsymbol \eta , \boldsymbol \pi ) = \{(\eta _{1},\pi _{1}),...,(\eta _{t},\pi _{t}),...,(\eta _{T},\pi _{T})\}\) be the histogram model for 𝜃 0, where π represents the occurrences of the movements (without duplicates) in 𝜃 0. Given its nature, π can serve as a proxy for τ. By contrast, to define feasible proxies for υ 1 and υ 2, we will make use of a procedure similar to the one adopted in the “spatial pyramid matching”, a pattern recognition technique which is used to recursively divide an image histogram into sub-histograms of features (Lazebnik et al. 2009). More precisely, let us define:

$$\begin{array}{ll} \mathcal{U} = (\mathcal{H}_{\theta} - \mathcal{H}_{\theta_{0}}) & = \{(\eta_{1},\lambda_{1}),...,(\eta_{t},\lambda_{t}),...,(\eta_{T},\lambda_{T})\}\\ & = \{(\eta_{1},\omega_{1} - \pi_{1}),...,\\&{\kern10pt}(\eta_{t},\omega_{t} - \pi_{t}),...,(\eta_{T},\omega_{T} - \pi_{T})\} \end{array} $$

and compute the median class η S of \(\mathcal {U}\). Next, we set:

$$\begin{array}{ll} & \mathcal{U}_{1} = (\boldsymbol\eta, \boldsymbol\lambda_{1}) = \{(\eta_{1},\lambda_{1}),...,(\eta_{s},\lambda_{s}),...,(\eta_{S},\lambda_{S})\}\\ & \mathcal{U}_{2} = (\boldsymbol\eta, \boldsymbol\lambda_{2}) = \{(\eta_{S+1},\lambda_{S+1}),...,(\eta_{t},\lambda_{t}),...,(\eta_{T},\lambda_{T})\} \end{array} $$

as sub-models associated with the semi-spaces of the target (C2 in Fig. 2) and competing cues (C1 in Fig. 2), respectively. Finally, the ensuing vectors λ 1 and λ 2 can be used as proxies for the quantities υ 1 and υ 2.

Finally, let:

$$ \begin{array}{ll} &\xi(\boldsymbol\tau) = [\phi(\boldsymbol\eta, \boldsymbol\tau)-\mathbf{1}] \log[\mathbf{1} - \phi(\boldsymbol\eta, \boldsymbol\tau)]^{T}\mathbf{1} \\ &\zeta_{1}(\boldsymbol{\upsilon}_{1}) = [\phi(\boldsymbol\eta, \boldsymbol{\upsilon}_{1})-\mathbf{1}] \log[\mathbf{1} - \phi(\boldsymbol\eta, \boldsymbol{\upsilon}_{1})]^{T}\mathbf{1} \\ &\zeta_{2}(\boldsymbol{\upsilon}_{2}) = [\phi(\boldsymbol\eta, \boldsymbol{\upsilon}_{2})-\mathbf{1}] \log[\mathbf{1} - \phi(\boldsymbol\eta, \boldsymbol{\upsilon}_{2})]^{T}\mathbf{1} \end{array} $$
(4)

being the cumulative residual entropies associated with the fast movements and the motor pauses, respectively, whereas τ, υ 1, and υ 2 are the associated T×1 unknown vectors.

Now, we have all the tokens to exactly define the non-linear optimization problem. In particular, by using the proxies π, λ 1 and λ 2, the NLP problem can be set as follows:

$$\begin{array}{@{}rcl@{}} &{\kern-6.5pt}\underset{\{\boldsymbol\tau, \boldsymbol{\upsilon}_{1}, \boldsymbol{\upsilon}_{2} \}}{\text{minimize}}& \mathbf{1}^{T} [\log(\boldsymbol\tau) \,-\, \log(\boldsymbol\pi)]\boldsymbol\tau \,+\, \mathbf{1}^{T}[\log(\boldsymbol{\upsilon}_{1}) \,-\, \log(\boldsymbol\lambda_{1})]\boldsymbol{\upsilon}_{1}\\ && +\, \mathbf{1}^{T}[\log(\boldsymbol{\upsilon}_{2}) - \log(\boldsymbol\lambda_{2})]\boldsymbol{\upsilon}_{2}\\ &{}\text{subject to:}& \psi = \xi(\boldsymbol\tau) + \zeta_{1}(\boldsymbol{\upsilon}_{1}) + \zeta_{2}(\boldsymbol{\upsilon}_{2})\\ && 1 \,=\, \mathbf{1}^{T}\boldsymbol{\tau}, \quad 1 \,=\, \mathbf{1}^{T}\boldsymbol{\upsilon}_{1}, \quad 1 \,=\, \mathbf{1}^{T}\boldsymbol{\upsilon}_{2} \end{array} $$
(5)

where 1 is a vector of ones of appropriate order whereas the objective function to be minimized is the well-known Kullback–Leibler (KL) divergence, which is expressed as a function of the unknowns quantities τ, υ 1, υ 2 and the proxies π, λ 1, λ 2. Note that the first constraint in the equation ensures that all the estimates are still compatible with the decomposition of the quantity ψ whereas the other constraints are just normalization factors which guarantee well-posed estimates.

Considering the nature of the problem (5), the (approximated) solutions can be found numerically using an interior-point method that allows for efficient and effective handling of the equality constraints described in Eq. 4 (Gill et al. 1981). A preliminary pilot study confirmed this expectation by showing that the algorithm was indeed fast and accurate enough to provide stable and convergent numerical solutions.

Implementation

The EMOT method is implemented as an add-on package for the Matlab environment and includes function for data modeling, estimating parameters, and performing graphical analyses of trajectories and extracted measures. All the features involved in the analysis (e.g., NLP algorithm and parameters, histogram parameters, sensitivity analysis) can be set by the user. Moreover, different types of analysis (e.g., subject-by-subject and global) with different characteristics can be selected. Finally, the application provides a single output containing all the results of the analysis. The EMOT tool works with the raw x-y computer-mouse trajectories that can be collected with any recording software. The EMOT package is freely available at http://polorovereto.unitn.it/~antonio.calcagni/emot.html.

Interpretation of the EMOT measures

Measure

ψ The cumulative residual entropy is used as an overall quantity of the spatial motion information expressed by the movement data. As such, ψ conveys the overall dynamics of the x-y trajectory executed in the movement space (e.g., degree of freedom of the movement, spatial exploration) and correlates to the amount of spatial uncertainty due to the movement dynamics. Consequently, it takes its minimum value when the movement is simply executed in the direction of the target C2, i.e., when the movement shows no attraction toward the competing cue C1, whereas takes its maximum value when the movement dynamics show attraction/competition toward the competing cue C1 (see Fig. 2a). This interesting property makes ψ directly comparable with MD/AUC measures of the DGA approach. However, since ψ is calculated using the models \(\mathcal {H}_{\theta }\) and ϕ(η,ω) rather than directly 𝜃, it is more robust to noisy and uneven movement trajectories than MD/AUC.

Decomposition of

ψ The additive decomposition, ψ = ξ + ζ 1 + ζ 2, allows for a simple and effective analysis of the motor information and provides insights into the mouse trajectories recorded during the task. Indeed, the measures ξ, ζ 1, and ζ 2 quantify motion events that are relevant in the mouse-tracking context, as they might reveal important aspects of the involved cognitive processes, such as decisional conflicts and decisional uncertainty (Johnson et al. 2012; Glaholt and Reingold 2011). In particular, the component ξ quantifies rapid movement events associated, for instance, with motor-execution behaviors, which can arise after a decision has been made. Instead, the components ζ 1 and ζ 2 represent pause movement events that are associated, for example, with processes of goal formulation and/or reformulation and can uncover the dynamics of the decision process (Johnson et al. 2012; Glaholt and Reingold 2011).

Two empirical applications

In this section, we will shortly present two applications of the EMOT approach based on two already published datasets. In the first application, we will describe a single-trial analysis on data coming from a standard categorization task (Dale et al. 2006). Here, we will evaluate (a) how the entropy-based indices behave and (b) how they can be interpreted when empirical movement profiles are considered individually. In the second application, we will also evaluate the EMOT measures on a group analysis of lexical decision data (Barca and Pezzulo 2012). In both the applications our measures will be also matched to a set of the most frequently used DGA measures. Note that the applications we report here have only illustrative purposes with the aim to highlight the importance of considering both the movement and pause measures as relevant indexes of cognitive processing.

Categorizing atypical exemplars

General context and motivation

Categorizing stimuli with atypical nature usually involves competition and uncertainty in participants that have to choose among categories with high similarities. Indeed, typical stimuli are also recognized faster and more accurately than atypical ones (Dale et al. 2007; Rips et al. 1973). By using the mouse tracking methodology, Dale et al. (2006) have investigated the effect of stimulus typicality in the context of the categorization of natural stimuli (e.g., animals). In particular, they proposed a simple semantic categorization task where basic level animal names (e.g., gorilla) were assigned to their respective superordinate category (e.g., mammal). Participants were asked to categorize animal stimuli in the correct category by choosing between two possible assignments (e.g., gorilla: mammal vs. reptile) with the hypothesis that stimuli showing higher proximity with the incorrect category will result in larger mouse trajectories reflecting competition and uncertainty in the categorization process. We expect that task competition and uncertainty would be reflected both in terms of fast movements and motor pauses.

Data, measures, procedure

The sample is represented by 34 right-handed students from the Cornell University. Lexical stimuli (125 names of animals) belonging to the natural superordinate categories of mammal, fish, reptile, bird, and insect, were extracted from the McRae et. al.’s study (McRae et al. 1997) together with their feature-semantic measures and measures of similarities. Participants were presented with two category names (target and competitor) placed in the right and left-hand corner of a computer screen whereas the stimulus name was placed on the bottom of the screen. Participants were asked to categorize the animals by clicking the computer-mouse on their correct categories. In the meantime, the x-y streaming of the mouse coordinates were recorded. The positions of target and competitor and, for each trial, the competitor category were randomly chosen among the four possible animal categories.

Data analysis and results

We re-analyzed the original x-y mouse trajectories data using the EMOT approach. In addition, we also computed some well-known DGA measures (MD, AUC, xFlips, samEnt: sample entropy, AC: acceleration components) using the package mousetrap developed by P. J. Kieslich and F. Henninger.Footnote 5 The AC measure was instead calculated as indicated in Duran et al. (2010).

Figure 3 shows a prototypical case, which can be nicely described by the DGA approach (e.g., see: Spivey & Dale, 2006). The pattern contains two small pauses at the beginning and end of the movement, possibly reflecting planning motor executions and their finalization (e.g., see: Flash & Henis, 1991). The movement appears simple and clean with total spatial information, which is entirely due to fast movements (ψ = ξ=2.37) with no motor pauses in the direction of the competing lexical category reptile. In this example, EMOT and DGA measures convey converging results.

Fig. 3
figure 3

First case study: Participant Id. no. 2. EMOT measures: ψ=2.37, ξ=2.37 (100 %), ζ 1=0 (0 %), ζ 2=0 (0 %). DGA Measures: MD=0.28, AUC=0.17, xFlips=1, samEnt=0.14, AC=12. Completing time: 650 ms. Average sample time: 845 ms. Note that \(\mathcal {H}_{\theta }\) and \(\mathcal {H}_{\theta _{0}}\) are the spatial motion models whereas τ, υ 1, and υ 2 represent the estimated components of the EMOT model

By contrast, Fig. 4 depicts a typical movement profile characterized by uncertainty in classifying the stimulus seal in the correct category mammal because of the lexical competition of the other category fish. This is reflected in the trajectory as well as in the pauses occurring toward the competing category. The empirical movement path is defined by two main features: a first rapid movement toward the incorrect category, which ends with a motor pause and a second rapid movement toward the correct category. The overall dynamics of the movement is largely due to the two fast movements (ξ=2.94, 91 %) with only a motor pause for the competing category (ζ 1=0.28, 9 %; ζ 2=0, 0 %). This profile is an example of a movement path composed by two discrete sub-movements separated by a pause (e.g., see: Hwang et al., 2005; van der Wel, Eder, Mitchel, Walsh, & Rosenbaum, 2009), which is difficult to reconcile with the assumption of continuous movement trajectories assumed by some of the DGA interpretations (e.g., see: J. Freeman, Dale, & Farmer, 2011).

Fig. 4
figure 4

First case study: Participant Id. no. 26. EMOT measures: ψ=3.22, ξ=2.94 (91 %), ζ 1=0.28 (9 %), ζ 2=0 (0 %). DGA measures: MD=0.59, AUC=0.16, xFlips=2, samEnt=0.26, AC=16. Completing time: 1100 ms. Average sample time: 1031 ms. Note that \(\mathcal {H}_{\theta }\) and \(\mathcal {H}_{\theta _{0}}\) are the spatial motion models whereas τ, υ 1, and υ 2 represent the estimated components of the EMOT model

Figure 5 illustrates a more complex profile with an atypical curvature toward the incorrect category fish. Here, the pattern shows a movement composed by three sub-movements with three areas of motor pause in the direction of both categories. The movement starts with a rapid ballistic trajectory toward the correct target mammal, turns toward the incorrect category fish, and turns again toward the correct mammal where it definitely stops. In this example, all the three of the EMOT measures were activated (ξ=1.65, 75 %; ζ 1=0.35, 16 %; ζ 1=0.18, 9 %). Note that for this pattern the DGA measures do not capture all the richness capitalized by the movement trajectory. For instance, AUC = 0 suggests that no attraction has been exercised by the competing category fish.

Fig. 5
figure 5

First case study: Participant Id. no. 19. EMOT measures: ψ=2.18, ξ=0.53 (24 %), ζ 1=1.64 (75 %), ζ 2=0.01 (1 %). DGA measures: MD=0.63, AUC=0, xFlips=3, samEnt=0.52, AC=25. Completing time: 1050 ms. Average sample time: 880 ms. Note that \(\mathcal {H}_{\theta }\) and \(\mathcal {H}_{\theta _{0}}\) are the spatial motion models whereas τ, υ 1, and υ 2 represent the estimated components of the EMOT model

Finally, Figs. 6 and 7 show two patterns of movements where there is no (continuous) attraction toward the distractor. In particular, Fig. 6 presents a path with a fast movement in the direction of the correct category mammal followed by a motor pause just before the final clicking. The overall movement dynamic is mainly due to fast motor executions (ξ=1.56, 81 %) and a motor pause in the area of the target (ζ 2=0.36, 19 %; ζ 1=0, 0 %). Note that in this case some of the DGA measures, such as MD and AUC, show negative values, which are hard to interpret in substantive terms. Moreover, xFlips suggests a kind of complexity along the x-axis although there is no displacement toward the category bird. Similarly, Fig. 7 depicts a movement path characterized by a sequence of submovements along the direction of the correct category mammal, conjoined by two pauses. The overall dynamics of the movement is mainly due to movement executions (ξ=0.78, 82 %). Motor pauses, which are in the direction of the target (ζ 2=0.13, 18 %; ζ 1=0, 0 %), may reflect here the activation of verification components (e.g., see: Meyer et al., 1988).

Fig. 6
figure 6

First case study: Participant Id. no. 18. EMOT measures: ψ=1.92, ξ=1.56 (81 %), ζ 1=0 (0 %), ζ 2=0.36 (19 %). DGA measures: MD=−0.20, AUC=−0.07, xFlips=2, samEnt=0.17, AC=19. Completing time: 900 ms. Average sample time: 817 ms. Note that \(\mathcal {H}_{\theta }\) and \(\mathcal {H}_{\theta _{0}}\) are the spatial motion models whereas τ, υ 1, and υ 2 represent the estimated components of the EMOT model

Fig. 7
figure 7

First case study: Participant Id. no. 20. EMOT measures: ψ=0.91, ξ=0.78 (82 %), ζ 1=0 (0 %), ζ 2=0.13 (18 %). DGA measures: MD=0.09, AUC=0.02, xFlips=4, samEnt=0.71, AC=23. Completing time: 1300 ms. Average sample time: 817 ms. Note that \(\mathcal {H}_{\theta }\) and \(\mathcal {H}_{\theta _{0}}\) are the spatial motion models whereas τ, υ 1, and υ 2 represent the estimated components of the EMOT model

A lexical decision task

In this second application, we studied the behavior of EMOT and DGA measures by looking at participants’ performance in a lexical decision task.

General context and motivation

Lexical decision is one of the most used task in psycholinguistic literature since the 1960s. Generally, in this task, participants see a printed stimulus on the screen (e.g., hand) and have to perform a dichotomous choice to decide whether it is a real word or not. The lexical decision task is widely used in the study of visual world recognition and reading. Since this task is very simple and versatile, it has been employed in tens of different experimental situations and is suitable for studying multiple levels of the linguistic processing (e.g., Hawkins, Brown, Steyvers, & Wagenmakers, 2012; Norris & Kinoshita, 2008; Yap, Balota, Tse, & Besner, 2008).

In this second application, we assessed the extent to which the overall entropy measure ψ and each of its subcomponents, ξ, ζ 1, and ζ 2, are affected by a stimulus type factor composed by words, pseudowords (i.e., sequences of letters that do not exist in the language but can be read), and random strings (i.e., random sequence of letters that are phonotactically illegal in the language) in a lexical decision task. We expect that the entropy measure ψ will be characterized by a pattern similar to that of the geometric measure MD/AUC with in general pseudowords being associated with larger levels of uncertainty in the decision process. Moreover, we also assessed the importance played by the pauses or pauses (measured by means of the quantities ζ 1 and ζ 2) in shaping the level of uncertainty associated with more difficult stimuli.

Data, measures, procedure

As a case study, we take advantage of the work by Barca and Pezzulo (2012) that is, for our purpose, a perfect case of lexical decision. They ran a lexical decision experiment in Italian by adopting a stimulus type factor with four different categories: Words of high written frequency (HF, e.g., acqua ’water’), words of low written frequency (LF, e.g., cervo ’deer’), pseudowords (PW, e.g., dorto), and strings (STR, e.g., btfpr). Participants saw a total of 96 stimuli (1/4 for each category), once at the time, and had to categorize each stimulus as either a word or a nonword; the two alternative answers were displayed at the top left and top right of the screen and the choice was performed by reaching one of the two alternatives with the mouse.

Data analysis and results

We re-analyzed the original x-y mouse trajectories collected by Barca and Pezzulo (2012) by using the EMOT approach and compared the entropy measures with the standard measures of the DGA (i.e., MD, AUC, xFlips, samEnt, and AC) that are computed as described for the first application. Moreover, in our analysis, other than using the stimulus type factor used by Barca and Pezzulo (2012), we added a further psycholinguistic variable, bigram frequency, as a covariate. Bigram frequency is the frequency with which adjacent pairs of letters (bigrams) occur in printed texts; for its characteristics, it may be considered as a measure of orthographic typicality (see, e.g.,: Hauk, Davis, Ford, Pulvermüller, & Marslen-Wilson, 2006) and we used it as a covariate in the analysis to further reduce error variance. Only bigram frequency was used since it was the only psycholinguistic variable that could be computed for all the stimuli.

Examples of single-case analysis

Figure 8 illustrates a profile from the low frequency (LF) condition. The path is a typical case characterized by a first fast movement that proceeds on the center of the screen for a while, then a second movement is initiated toward NW before reversing in the direction of W. This pattern suggests a prolonged decision uncertainty rather than a continuous competition between the two categories. The overall movement dynamics is proportionally due to motor executions (ξ=2.37, 62 %) and pauses (ζ 1=1.08, 28 %; ζ 2=0.34, 10 %). Figure 9 shows instead a quasi-cyclic trajectory involving several discrete sub-movements and pauses with no attraction in the direction of NW. The overall movement dynamics is mainly ascribed to motor executions (ξ=2.21, 79 %) and pauses toward the target (ζ 2=0.58, 21 %; ζ 1=0, 0 %). Interestingly, in this case, DGA measures fail to recognize the characteristics of this profile: AUC is close to zero because of the quasi-cyclic trajectory shown by this profile, which yields summing positive and negative portions of area at the same time.

Fig. 8
figure 8

Second case study: Participant Id. no. 25. EMOT measures: ψ=3.79, ξ=2.37 (62 %), ζ 1=1.08 (28 %), ζ 2=0.34 (10 %). DGA measures: MD=1.51, AUC=1.03, xFlips=10, samEnt=0.34, AC=23. Completing time: 1763 ms. Average sample time: 1509 ms. Note that \(\mathcal {H}_{\theta }\) and \(\mathcal {H}_{\theta _{0}}\) are the spatial motion models whereas τ, υ 1, and υ 2 represent the estimated components of the EMOT model. Note also that NW and W indicate nonwords and word categories whereas ghiro is the Italian stimulus for dormouse

Fig. 9
figure 9

Second case study: Participant Id. no. 2. EMOT measures: ψ=2.79, ξ=2.21 (79 %), ζ 1=0 (0 %), ζ 2=0.58 (21 %). DGA measures: MD=−0.26, AUC=0.01, xFlips=8, samEnt=0.26, AC=13. Completing time: 1794 ms. Average sample time: 1450 ms. Note that \(\mathcal {H}_{\theta }\) and \(\mathcal {H}_{\theta _{0}}\) are the spatial motion models whereas τ, υ 1, and υ 2 represent the estimated components of the EMOT model. Note also that NW and W indicate nonwords and word categories

Finally, Fig. 10 depicts a profile characterized by three submovements with two pauses in the direction of both the cues. The three movement executions represent a large amount of total spatial information (ξ=1.00, 57 %) whereas it is interesting to note that motor pauses are approximately the same in their values (ζ 1=0.379, 22 %; ζ 2=0.381, 21 %). This suggests that the participant showed greater uncertainty in categorizing the stimulus as nonword or word. Indeed, he/she first hesitated toward W, then toward NW, before selecting the correct final choice. In sum, all the three examples illustrated in this second application would result in some level of interpretation difficulty for the basic DGA measures (particularly for MD and AUC).

Fig. 10
figure 10

Second case study: Participant Id. no. 8. EMOT measures: ψ=1.76, ξ=1.00 (57 %), ζ 1=0.38 (21 %), ζ 2=0.38 (22 %). DGA measures: MD=0.70, AUC=0.38, xFlips=11, samEnt=0.14, AC=17. Completing time: 1263 ms. Average sample time: 1532 ms. Note that \(\mathcal {H}_{\theta }\) and \(\mathcal {H}_{\theta _{0}}\) are the spatial motion models whereas τ, υ 1, and υ 2 represent the estimated components of the EMOT model. Note also that NW and W indicate nonwords and word categories

Group analysis

As a first step, we ran nonparametric correlation analyses to evaluate the eventual degree of similarity between the entropy measures and the geometric measures. Table 1 reports the results of the correlational analyses. For this type of data, a high degree of similarity was observed between ψ and MD (and to some less degree between ψ and AUC), whereas only mild correlations were observed between ψ and the other DGA measures (xFlips, samEnt, and AC). Because of the model definition, we naturally observed also positive correlations between ψ and its subcomponents, ξ,ζ 1, and ζ 2, with ξ showing the largest correlation among the three terms. Interestingly, the correlational analysis showed a negative association between the two pause entropies ζ 1 and ζ 2. Finally, the last line of Table 1 reports the nonparametric correlations between the overall response times (RTs) and the mouse tracking measures. From a quick inspection of these correlations, it seems clear that the mouse tracking measures and RTs are independent quantities characterizing different components of the process. In particular, the overall duration of a lexical decision task does not capture the motor uncertainties unfolding during the decisional process.

Table 1 Spearman’s correlations for all possible distinct pairs of measures

As a second step, to analyze the impact of the experimental factor on the ψ measure (as dependent variable), we ran a robust linear mixed modelFootnote 6 (RobLMM) using stimulus type as factor (with HF as reference level of the factor) and bigram frequency as covariate; random intercepts for subjects and items were also included in the model. Table 2 reports some descriptive statistics of the subcomponents of ψ as a function of the four levels of the experimental factor. To compare the results for ψ, we also run similar analyses using the DGA measures as new dependent variables. Table 3 reports the overall results of the RobLMM analysis. As expected, the results of the robust regressions showed very similar patterns for AUC, MD, and ψ: In all the models, there was a significant difference between PW and HF (AUC: β pw=0.123, SE = 0.016, t=7.655; MD: β pw=0.233, SE = 0.018, t=13.212; ψ: β pw=10.574, SE = 0.832, t=12.714) with an increasing degree of uncertainty in PW with respect to HF. However, ψ showed also marginal effects for STR (β str=2.231, SE = 1.117, t=1.998). No further effect reached significance in the three models. By contrast, sample entropy and acceleration showed a moderate effect between LF and HF (samEnt: β lf=0.012, SE = 0.006, t=2.034; AC: β lf=0.015, SE = 0.006, t=2.488) but only sample entropy was significantly different between PW and HF (samEnt: β pw=0.015, SE = 0.006, t=2.488). The last DGA measure, xFlips, did not show any significant difference. Finally, the results for the RT analysis showed a significant difference between PW and HF (β pw=162.011, SE = 9.575, t=16.919) as well as a significant difference between LF and HF (β lf=41.076, SE = 9.157, t=4.486). In particular, this latter result about low frequency words showed how response time and some of the mouse trajectory measures, such as the EMOT measures, MD and AUC, captured different facets of the lexical decision process.

Table 2 Mean values of the entropy, geometric, and response time measures as a function of the experimental factor
Table 3 Summary of the main effects observed on the different measures

After having evaluated the overall entropy effect as measured by ψ, we further investigated the single components, ξ,ζ 1, and ζ 2, of the EMOT model by running different RobLMMs on the same set of independent variables. About the entropy measure of fast movements, ξ, the result of the RobLMM analysis showed a significant difference between PW and HF (β pw=9.625, SE = 0.772, t=12.461), which was also the only observed statistically significant result in the model. The pattern was similar to that of the overall measure ψ, with a larger degree of uncertainty in PW as compared to HF. For the pauses toward the distractor, ζ 1, the results of the robust analysis showed a significant effect for both PW and STR (β pw=0.282, SE = 0.054, t=5.222; β str=0.207, SE = 0.073, t=2.856) with larger levels of uncertainty in PW and STR with respect to HF. By contrast, the pauses toward the target (ζ 2) did not show any significant effect in the RobLMM analysis. Overall, the pattern of results showed that stimulus type affected two of the three components of the overall entropy measure ψ. In particular, the level of entropy is larger for PW than HF both in pure movements and pauses toward the distractor (ξ and ζ 1); by contrast, stimulus type does not affect the level of entropy for pauses toward the target, ζ 2. These results indicate that in some psycholinguistic tasks, such as lexical decision with words, pseudowords, and random strings, pure movements play a relevant role in characterizing the decisional process, with in general easier stimuli being associated with lower levels of movement entropy as measured by mouse trajectories. However, the uncertainty related with pure movement does not tell the entire story, as also the entropy associated with pauses, in particular that occurred in the direction of the distractor, is an important aspect in describing the mechanisms involved in the decisional task. As far as we know, this is the first time that motor pauses have been modeled and investigated to better understand the cognitive processes involved in such empirical paradigms.

Discussion

The new EMOT approach aimed to analyze computer-mouse trajectories recorded during mouse-tracking experiments, in terms of their constitutive components (fast movements and motor pauses). EMOT offers a way not only to detect such components, but also to quantify them in terms of entropy, a means that helps to measure decisional uncertainty during two-choice categorization tasks (see Application 1). This is an important feature of EMOT, which is different from other existing approaches, such as DGA (Freeman and Ambady 2010) or parsing based techniques (Hwang et al. 2005).

In general, the decomposition of ψ entails a set of relations among the subcomponents ξ, ζ 1, and ζ 2 that can guide a researcher to interpret EMOT results. The first component ξ is related to (rapid) movements. Because two-choice categorization tasks naturally require a final decision, a good sampled trajectory should always show positive values of ξ, which indicates that a dynamic movement occurred. Even very fast movements always involve a number of motor executions that are detected by the subcomponent ξ. Consequently, movement profiles with ξ=0 would reveal problems during sampling of computer-mouse movements. Given its nature, ξ is similar to the more traditional DGA measures and may be associated to both the decisional unfolding and fulfillment in categorization tasks (e.g., see: Freeman & Ambady, 2010).

With regards to ζ, positive values are not mandatory events, as indicated by those profiles showing rapid motor execution without relevant slowing-down phenomena (e.g., see Fig. 3). This could be the case, for example, when the decision process either has been completed before a participant moves the pointer or the decision unfolds smoothly. However, the most typical profiles involve positive values for ξ followed by positive values for one of the two components ζ 1 or ζ 2. We suggest that ζ 1 and ζ 2 mirror different processes. A prominence of ζ 1 may be associated to a conflict between the target and the competitor category, with larges values denoting a stronger competition exerted by the distractor (Fig. 4 exemplifies this process). Differently, a prominence toward ζ 2 seems to be associated to the verification component of the decision process, which may be tentatively linked to the activity of the monitoring system (e.g., see: Resulaj, Kiani, Wolpert, & Shadlen, 2009).

In sum, the EMOT approach as well as its empirical validation give us the chance to make a more general consideration on the dual-choice task. As nicely exemplifies Fig. 5, the categorization process does not necessarily involve a continuous mouse attraction toward the distractor; instead, in some cases, the process seem to unfolds through discrete consecutive movements (e.g., see: Fishbach, Roy, Bastianen, Miller, & Houk, 2007).

For an overall schema of the most relevant cases of EMOT measure combinations, see Table 4.

Table 4 Exemplary relations among ξ, ζ 1, ζ 2 and their interpretations

Potential and limits of the EMOT approach

Although temporal analyses might shed light on the dynamics of the cognitive characteristics of the movement (Jonsen et al. 2005), we recognize that the EMOT framework uses the spatial components of empirical trajectories as the only informational source. However, this simplification can be easily justified. First, we were interested in analyzing spatial descriptors of the movement, which are important in the two-choice categorization domain (like cue attraction). This entailed the use of a trajectory-oriented approach of analysis, focusing on the extraction of specific location parameters (e.g., absolute or relative angles) used in subsequent statistical analyses (Demšar and Virrantaus 2010). Second, as also pointed out by the results of our second application, the analysis of the spatial information provided different insights into the mechanisms underlying mouse-tracking tasks, which are not always uncovered by the temporal information (Georgopoulos et al. 1981). Moreover, the use of more complex models considering both spatial and temporal information together, such as the correlated random walk (CRW), usually requires some additional assumptions regarding the movement generation phase and the type of movement structure (e.g., navigation with no references, movement patters with no repetition. See: Bergman, Schaefer, & Luttich, 2000; McClintock et al., 2012), which would be difficult to test in the mouse-tracking context. By contrast, EMOT does not require specific assumptions as it provides a descriptive/phenomenological representation of the process. Indeed, it offers a simple mathematical method to summarize the movement trajectories with a small set of spatial descriptors and measures.

Like for all mouse-tracking methods, also the quality of EMOT results are strictly related to those of the original x-y data registration. Although computer-mice have notable advantages (e.g., low cost, easy availability), they have limitations that may cause problems to the kinematic properties of the trajectories (O’Really and Plamondon 2011). Because some setting variables, such as mouse gain, handedness, arm positioning, can affect results of mouse-tracking analyses, computer-mouse devices should be carefully used and possibly calibrated before running experimental tasks (Flodgren et al. 2007; O’Really and Plamondon 2011). For instance, in a two-choice categorization task, as those discussed in this paper, mouse gain may affect the reaching trajectory of the mouse by intensifying or diminishing the level of smoothness of the trajectory. For example, cursor speed may deteriorate the fine-grained movements of the mice. Consequently, eventual competition/curvature of the trajectories could result in artifacts arising from the lack of precision of the hand. Indeed, as is known, larger movement amplitudes are usually required by low mouse gain whereas a higher precision is instead needed with high mouse gain (Dillen et al. 2005). Other factors such as cues direction, inter-distances among cues, size/width of the screen should be carefully checked since they may affect the quality of computer-mouse recordings (Phillips and Triggs 2001). Also, the age of participants should be considered before performing a computer-mouse task because aging may seriously affect motor control of the hand by reducing precision and smoothness of the movement (Smith et al. 1999). Consequently, also EMOT results may be influenced by kinematic characteristics, especially for the quantification of fast movements and pauses. However, since EMOT only works with x-y spatial information, kinematic variables that may affect the temporal resolution of the data (e.g., gain, acceleration) do not largely affect its measures. With this respect, a first simulation study conducted by varying acceleration, mouse gain, and direction of movements confirmed such intuition (see Supplementary materials). Anyway, as for other mouse-tracking based analytic techniques, setting computer-mouse devices in an appropriate way would certainly improve the general performances of the EMOT representation and reduce the effect of artifacts on the quantification of movement features such as pauses.

Despite some potential negative impact of uncontrolled kinematic variables, several advantages can be still advocated for the EMOT approach. For instance, comparing with DGA, it extracts movement features using a trial-by-trial approach and, consequently, the spatial measures are computed using all the information available in the empirical trajectories as they are recorded. An additional advantage of our method concerns the comparison of different movement profiles. For example, when researchers want to analyze whether two or more movement paths differ from one another in terms of spatial information. While this is impractical for other data-driven approaches, in our proposal different profiles can be efficiently compared in terms of cumulative Kullback-Leibler entropy (Baratpour and Rad 2012). For instance, given two movement profiles ϕ 1 and ϕ 2, which are associated with two movement data 𝜃 1 and 𝜃 2, their divergence can be calculated as χ(ϕ 1,ϕ 2)=[δ− log(δ)−1]T(1ϕ 1), where the vector δ is obtained from the element-wise ratio between the vectors 1ϕ 1 and 1ϕ 2 (with 1 being a vector of ones of appropriate order). This measure reaches its minimum value (χ=0) when the movement profiles ϕ 1 and ϕ 2 are the same, that is to say when the two movement paths provide the same quantity of spatial information.

Further extensions

As for any method, EMOT might be improved in various aspects. For example, the extension of our approach to multi-choice contexts, where participants have to assign stimuli to several possible categories. In a similar way, EMOT could be modified to handle with the problem of multi-trial integration, where entropy measures would need to be extracted from the distributions of the participants’ trials. This latter would turn out to the modelization of the error components across trials and participants, which would require the use of a stochastic decomposition of the equality ψ = ξ + ζ 1 + ζ 2 instead of the deterministic representation used throughout the paper. All these points need to be considered in further investigations.

Conclusions

In this paper, we described a novel approach to analyze spatial data arising from mouse-tracking experiments and offer a way to extract spatial descriptors of the movement in the context of two-choice categorization tasks. In particular, our contribution extended the main findings of the descriptive geometric approach (DGA) by providing a way to decompose the spatial information provided by the movement trajectories. We argued that mouse movement trajectories can be regarded as composed of fast movement executions and motor pauses, which are both important for a full understanding of the processes underlying a mouse-tracking task. To illustrate the utility of the new method, we applied our new approach to two real datasets of mouse-tracking data, representing typical categorization tasks in a two-choice paradigm. As expected, we were able to identify interesting spatial movement features, describing characteristics of the processes involved in this type of tasks, such as the decisional uncertainty. Moreover, the second application showed how the proposed entropy measures can capture facets of the categorization process, which would not be represented using standard RT as well as other geometric or distance-based measures. Overall, these results highlighted how differences in terms of global movement trajectories can be better explained considering their constituent components (i.e., fast movement and pauses).

In conclusion, this study has yielded valuable findings in the decomposition of movement trajectories in terms of fast movements and pauses. Indeed, the information related with the sole movement does not tell the entire story, as also pauses, in particular those occurring in the direction of the competing cue, are important aspects in describing the mechanisms involved in decisional tasks. As far as we know, this is the first time that motor pauses have been modeled and investigated to better understand the cognitive processes involved in mouse-tracking studies. We believe that new research exploring and analyzing movement trajectories will benefit from the advantages gained by the EMOT method.