Evidence that within-dimension features are generally processed coactively

Blunden, Anthea G.; Howe, Piers D. L.; Little, Daniel R.

doi:10.3758/s13414-019-01775-8

Evidence that within-dimension features are generally processed coactively

40 Years of Feature Integration: Special Issue in Memory of Anne Treisman
Published: 28 June 2019

Volume 82, pages 193–227, (2020)
Cite this article

Download PDF

Attention, Perception, & Psychophysics Aims and scope Submit manuscript

Evidence that within-dimension features are generally processed coactively

Download PDF

Anthea G. Blunden¹,
Piers D. L. Howe¹ &
Daniel R. Little¹

1083 Accesses
3 Citations
Explore all metrics

Abstract

In this paper, we examine whether information about an item’s category, provided by the same dimension type presented across multiple spatial locations (which we term within-dimension features), is processed independently or pooled into a common representation. We use Systems Factorial Technology (SFT; Townsend & Nozawa, Journal of Mathematical Psychology, 39, 321–340, 1995) and fit parametric logical rule-based models to diagnose whether information processing is serial, parallel, or coactive. The present work focuses on expanding the scope of categorization response time (RT) models by synthesizing recent work in perceptual categorization with theories of visual attention. Our results show that for the majority of participants, processing occurs coactively (i.e., is pooled into a single decision process). For the remainder, other processing strategies were found (e.g., parallel processing). This finding provides new insight into decision-making using within-dimension features presented in multiple locations. It also highlights the importance of both featural information and spatial attention in categorization decision-making.

Context, structure, and informativeness judgments: An extensive empirical investigation

Article 08 July 2020

Assessing information integration processes: a comparison of findings obtained with between-subjects designs versus within-subjects designs

Article 09 October 2017

A Tale of Two Literatures: A Fidelity-Based Integration Account of Central Tendency Bias and Serial Dependency

Article 03 January 2022

Every day we make decisions by identifying, discriminating, comparing, and categorizing objects that have different visual features which come from the same dimension (e.g., colors, sizes, shapes etc.). To give a few examples of this everyday decision-making, imagine you wish to select a ripe banana from a bunch: How do you decide which banana is the most preferable? An obvious approach is to compare the color of the bananas, preferring the bright yellow bananas while avoiding the green and overripe brown bananas. Likewise, a food safety inspector may need to comply with federal guidelines on the color of meat in order to decide whether the rib eye is safe, and must therefore compare the color of the rib eye with a meat safety color chart. A navy ship office may need to identify and interpret the color combination of the signal flags of an approaching vessel. Understanding how we make categorization decisions using stimuli that vary on the same type of feature is a fundamental question in the psychology of perception and cognition. Our interest in these examples and the present paper is not how we identify the color, but rather how that color satisfies some criterion that informs some decision: Is this the ripest banana? Is the meat fresh enough? What is the other ship’s intention?

Recent work has utilized rule-based theories of categorization incorporating theories of response time (RT), not only to characterize decision-making, which necessitates the integration of a range of different features (e.g., size, color, shape) and their configurations, but also to answer more fundamental questions regarding perception, attention, and decision-making (Griffiths, Blunden, & Little, 2017; Little, Nosofsky, & Denton, 2011; Fifić, Little, & Nosofsky, 2010; Little, Nosofsky, Donkin, & Denton, 2013; Moneer, Wang, & Little, 2016). One critical question regards the underlying architecture of decision-making. Here, architecture refers to the organization of mental processes, or the way in which we combine information (Kantowitz, 1974; Sternberg, 1969; Schweickert, 1993; Townsend, 1984). More technically, it refers to distinguishing between serial, parallel, and coactive processing. For example, when selecting a banana do you make a decision on the color of each banana one at a time (i.e., serially)? Or alternatively, do you do so simultaneously, in separate decision-making channels (i.e., in parallel), or do you pool all your perceptual information into one single decision-making channel and make one overall decision (i.e., by processing the colors coactively; (Miller, 1982); see Fig. 1)?

Beyond identifying the processing architecture, which is not always straightforward due to pervasive model mimicry in many tasks (Townsend, 1990), a further question is what factors affect the processing architecture of decision-making? For example, characteristics such as whether or not features or dimensions can be attended to in isolation, and the configurations of these dimensions, such as spatial separation, have been shown to play a key role in determining how information from different dimensions is integrated (Moneer et al., 2016; Little et al., 2011). For example, it is difficult to attend to hue, brightness, or saturation independently. It would be therefore impossible to judge the ripeness of a banana based on the hue of its color alone, while ignoring saturation or brightness. Instead, information about these three dimensions must be pooled in order to make a categorization decision. On the other hand, judging which variety of bananas to buy (e.g., the Cavendish or the Lady Fingers, two popular Australian varieties) might require comparison of two separate dimensions, which are easy to attend to in isolation: freshness and price. Such information is likely to be located in two different spatial locations requiring attention to be independently deployed to each, with categorization decisions made independently. The ability to diagnose the underlying processing architecture is therefore crucial as it provides information about the role of spatial attention and selective attention in the sequencing of dimensions for processing. Here we use spatial attention to refer specifically to the allocation of attention across space, while selective attention refers to the more general process of weighting specific stimulus attributes, potentially including distance. Many paradigms for studying attention focus on accuracy, choice data, or mean RT alone (for example, spatial cuing tasks, see e.g., Posner, Snyder, & Davidson, 1980, or visual search, see e.g., Treisman and Gelade (1980)); however, these paradigms are often limited in their ability to differentiate serial, parallel, and coactive architectures from each other.

We seek to add to the growing body of work on visual processing in categorization by testing an important configuration of multidimensional stimuli such as those in the introductory examples: stimuli which are composed of dimensions separated in space but which comprise differing levels of the same type of feature, henceforth referred to as within-dimension features (see Fig. 2 for some examples). This type of stimulus is interesting from the perspective of visual attention as it has features that would appear on the same feature map, for instance, in Treisman and Gelade’s (1980) Feature Integration Theory (FIT). In our experiments, the features are separated spatially and take on different luminance values but can be continuously transformed from one to another. By contrast, this would not be the case for features conceived to be on different feature maps (e.g., shape vs. color), which we term between-dimension features (see Fig. 2). This is a key distinction in many theories of visual search, which find differences in performance between the two stimulus types (Wolfe et al., 1990). It is therefore possible that FIT may foreshadow a difference in categorization strategy for between and within-dimension stimuli.

Theories of visual attention such as FIT (Treisman & Gelade, 1980) and Guided Search (Wolfe, 1994a, 2007) provide useful insight into how attention operates as a function of which features are present in the visual scene. Although these theories do not explicitly address categorization, they propose that focused visual attention is driven by an early pre-attentive parallel processing stage. In this stage, different visual dimensions (e.g., luminance, orientation, spatial frequency, etc.) are registered separately and only later combined or bound to form a visual representation of an object or visual scene. Guided Search, for example, suggests that different dimensions are represented in separate maps which are summed together to form a master salience map that subsequently guides attention to relevant areas of a visual scene. Because of this guidance, when a target does not share any features with the distractors (e.g., a red square among green squares), search occurs efficiently, and purportedly in parallel. In these cases, the target “pops out”, and search time is generally fast and independent of the number of distractors. If, however, there is not a sufficient difference between the target and the distractors to cause “pop out”, search is not efficient (see e.g., Duncan & Humphreys, 1989) and is instead driven by effortful, attentive processing, which is purportedly more serial in nature (Wolfe, 1994a, 1998, 2007). Importantly, these theories emphasize attention as being driven by the specific features involved in the task, and therefore these tasks are often used to infer how features are processed. Given that we are using within-dimensions features, we may expect that these are combined pre-attentively into the same feature map, which then provides a single signal for decision-making. This would mean decision-making would occur coactively. Independent processing (e.g., serial or parallel) may be more consistent with between-dimension stimuli from different feature maps (indeed, this is what we see in the categorization literature; see e.g., Fifić et al.,, 2010).

To assess the processing of within-dimension features in perceptual categorization, we utilized Systems Factorial Technology (SFT; Townsend & Nozawa, 1995; Little, Altieri, Fifić, & Yang, 2018). Specifically, we investigated stimuli of opposite luminance polarity in separate locations of a visual display in a categorization decision-making task. Several previous studies of visual attention (Duncan & Humphreys, 1989; Wolfe et al., 1990; Mordkoff & Yantis, 1993) have also examined within-dimension features, and we return to these studies in our discussion; we first review related work in perceptual categorization and then present our current experiments.

Dimensional processing in categorization

The diagnosis of processing architecture during decision-making using visual information has been the focus of recent work in categorization (Blunden, Wang, Griffiths, & Little, 2015; Cheng, Moneer, Christie, & Little, 2017; Fifić et al., 2010; Little et al., 2011, 2013; Moneer et al., 2016). This literature draws on theories of visual attention to investigate how information from multi-dimensional sources is integrated during decision-making. The goal of this work is to provide a quantitative and detailed diagnosis of mental architecture using the logical rules modeling framework (Fifić et al., 2010), which complements the SFT analyses (Townsend & Nozawa, 1995; Little et al., 2018) in order to yield patterns of RTs across the entire time course of information processing.

To date, this work has focused on the integration of information from between-dimension features and has been successful in characterizing the underlying architecture of this process using a variety of different dimension types and configurations. Most relevant to the current work is the seminal paper by Fifić et al., (2010) introducing the logical rule models. Fifić et al., (2010) presented participants with two rectangles separated in space, one of which varied in saturation, and the other of which varied in the position of a line contained within the rectangle. Because these dimensions could easily be attended to independently (i.e., because they are separable—see e.g., Garner and Felfoldy, 1970, Garner, 1974—and physically separated), these dimensions could be processed in serial when participants were instructed to do so. Serial processing was also found for spatially separate dimensions when participants were allowed to adopt their own categorization strategies (Little et al., 2011). However, when two separable, between-dimension features were presented in a spatially overlapped fashion (Experiment 2, Little et al., 2011), processing was more parallel.

These findings are interesting as they highlight the importance of spatial configuration and attention in determining the architecture that underlies categorization. Before the introduction of the logical rule models, all models of categorization RT, including the successful exemplar-based random walk model (Nosofsky and Palmeri, 1997), stochastic general recognition theory (GRT; Ashby, 2000), and decision-bound models (Ashby, Boynton, & Lee, 1994; Maddox, 1992; Maddox & Ashby, 1996), assumed that the features of objects were pooled together into single objects (i.e., which we term coactivity). In both of the cases described above, both used separable dimensions, but having features positioned in separate locations induced serial processing, whereas overlapping the features in space produced more parallel processing. This would suggest that feature type is not necessarily a key component affecting processing strategy in categorization decision-making tasks but rather that location of the features is primary. Following this logic, we would expect our within-dimension luminance stimuli to require selective attention to resolve the feature values at each location, and consequently, categorization decision should proceed serially with separate micro-decisions made at each location combined using logical rules to determine the final response.

On the other hand, there are likely additional factors that may determine how features are processed. For instance, not all features can easily be attended to independently or selectively (Garner, 1974; Shepard, 1987; Nosofsky, 1988); these types of features, termed integral features, are thought to be processed holistically or configurally (e.g., hue, saturation, and brightness of colors in the Munsell color space; Lockhead & King, 1977; Nosofsky, 1988). Using the same categorization methodology, Little et al., (2013) found that the categorization of colors varying in saturation and brightness was best described by a coactive process (see also Blunden et al., 2015). Instead of making decisions separately along the brightness dimension and saturation dimension, participants instead pooled information about these dimensions into a single decision-making channel. In contrast to Fifić et al., (2010) and Little et al., (2011), the type of dimension—that is, the fact that the dimensions were integral rather than separable—played a key role in determining the processing architecture.

Integral dimensions necessarily occupy the same spatial location. However, Moneer et al., (2016) investigated categorization of whole-object features, which are features that comprise an entire object, and therefore spatially co-located, but are notionally separable, such as shape and size or shape and color. These whole-object features are between-dimension features in the sense that they would activate different feature maps,^{Footnote 1} but, despite being comprised of separable features, these features have been traditionally been characterized as integral (Biederman and Checkosky, 1970; Smith & Kilroy, 1979). Using the logical rules paradigm, Moneer et al., (2016) showed that these dimensions elicit independent, multichannel processing (i.e., serial or parallel processing). Similar results have been found with composite faces, which have also been traditionally treated as holistic (Cheng, McCarthy, Wang, Palmeri, & Little, 2018). Hence, the processing of different feature types not only in the same spatial location but comprising the whole object, depends on whether those features are separable or integral.

A natural question arising from these experiments concerns the categorization of within-dimension but spatially separated features of the type which are often used in studies of visual search (e.g., a red pop-out target in a field of green distractors). In the present paper, we use the strong inferential methods to answer this question. Within-dimension stimuli, such as luminance discs, provide an important point of investigation. Likely, because these within-dimension features are presented in separate locations, selective attention will be required to resolve the feature values. Consequently, spatial configuration will play a stronger role in determining processing strategy, and we expect processing to proceed serially as a result. On the other hand, these luminance values would be represented by the same feature map (Treisman & Gelade, 1980) and so it is possible that they would instead be processed coactively. While parallel processing seems somewhat unlikely, it is nonetheless worthy of consideration as a possible means for processing the within-dimension features. For example, in simple redundant target detection of two luminance targets, processing appears to proceed in parallel but with limited capacity (Townsend & Nozawa, 1995; Yang, Little, & Hsu, 2014). Visual attention, stimulus configuration, and dimension type, may all play key roles, and none of the candidate models (serial, parallel, and coactive) can be ruled out a priori. In the remainder of the introduction, we provide a detailed explanation of the categorization decision-making paradigm and logical rule models which we utilize to uncover how within-dimension features are processed.

Logical rules design

In order to differentiate the processing architectures, we utilize a design that provides strong diagnostic contrasts between the predictions of each of the candidate architectures. It is useful to describe these predictions with reference to Fig. 3, which shows a variant of the double factorial design proposed by Fifić et al., (2010). In this design, individual stimuli are comprised of the orthogonal combination of two dimensions, each of which vary over three levels. This generates nine stimuli, each of which comprise different levels of the two dimensions. While the double factorial paradigm has previously been implemented using single items which comprise both dimensions (e.g., halves of a face; Cheng et al., 2018, or two parts of a lamp; Fifić et al.,, 2010) the within-dimension feature stimuli used in the present experiment have different values of the same feature at different respective locations in space. Hence, each stimulus comprises a pair of discs, with each disc varying on three levels of saliency with respect to the background.

The upper-right quadrant comprises the target category (category A) stimuli. The dotted line represents the category boundary between the target category and the contrast category (category B). Items that lie closer to the category boundary should be more difficult to discriminate (Ashby & Gott, 1988; Nosofsky, 1986); hence, stimulus dimensional values of either high discriminability (H) or low discriminability (L) combine to form four stimuli which are defined by their relative difficulty in discriminability: HH, HL, LH, and LL.

The contrast category stimuli are also identified by their location in the category space. The redundant stimulus, R, satisfies both of the boundary decisions necessary to classify a stimulus as belonging to the contrast category (i.e., it is both to the left and below the decision boundary). The stimuli adjacent to R are termed the interior stimuli, I_X and I_Y, whereas the stimuli at the far edges are termed the exterior stimuli, E_X and E_Y.

In order to correctly classify a target category stimulus, a conjunctive rule on both dimensions must be satisfied. That is, a stimulus must have a value on both dimension X and Y that exceeds the horizontal and vertical decision boundaries, respectively. Specifically, the luminance of both disks must have a value of 2 (medium luminance) or higher. Hence, stimuli from the target category must be processed exhaustively (both dimensions must be processed before a correct decision can be made). Stimuli belonging to the contrast category can be correctly classified using a disjunctive rule (i.e., stimuli need only be below or to the left of the horizontal and vertical decision boundaries, respectively).

Note that these rules, used to instantiate the categories in the task, do not presume any sort of processing architecture. The fact that both dimensions must be processed exhaustively to correctly classify a target category stimulus, does not preclude this processing from being carried out one dimension at a time in serial, or simultaneously in parallel, or indeed pooled into a single integrated percept. The following section describes how the predictions from each of these models (and combinations of stopping rules) varies across both categories. The important point to note is that the model predicts that processing will be exhaustive for the target category when responding is correct, but processing may be self-terminating or exhaustive for the contrast category. Of course, a participant might self-terminate and still be accurate when responding to a contrast category item.

Target category predictions

Discriminating target category discs that are close to the decision boundary should be slower than discriminating discs that are further away (Ashby & Gott, 1988). The way in which these discriminations are combined varies for each model, and qualitatively different mean RTs are predicted by each model architecture for the target category stimuli (shown in the left panel of Fig. 4). These RTs are readily summarized by the mean interaction contrast (MIC). The MIC is calculated by finding the difference between the difference of the low and high discriminability values on one dimension and the difference of the low and high discriminability values on the other dimension:

$$ MIC = (RT_{LL} - RT_{LH}) - (RT_{HL} - RT_{HH}) $$

(1)

Serial models predict an additive pattern of mean RTs (MIC = 0), parallel models predict an under-additive pattern of mean RTs (MIC < 0), and coactive models predict an over-additive pattern of mean RTs (MIC > 0; Townsend & Nozawa, 1995). A brief explanation for these predictions is outlined below; however, for a more detailed outline, please see Fifić et al., (2010).

Serial models predict an additive pattern as both LH and HL items will show some slowing relative to the HH item due to their lower discriminability on one of the dimensions. The increase of RT for the LL item compared to the HH item is simply the sum of the individual sources of slowing. Parallel models predict an under-additive pattern because the RTs for the target category items are determined by the slower of the two decisions (i.e., the maximum processing time). The LH and HL stimuli will be therefore much slower than the HH stimuli. The LL stimulus will be only slightly slower than either the LH or HL stimuli. Finally, Townsend and Nozawa (1995) provide a mathematical proof demonstrating coactive models result in an over-additive pattern of results. This finding has been corroborated by simulations done by Fifić, Nosofsky, and Townsend (2008a). The predictions outlined above are non-parametric in that they do not depend on the particular forms of the RT distributions; hence, the qualitative contrasts apply to the entire class of serial, parallel, and coactive models.

Further diagnostic evidence for processing architecture from the target category can be found by calculating the survivor interaction contrast (SIC). The SIC is calculated using the survivor function for each stimulus, at each time value, t:

$$ SIC(t) = [S_{LL}(t) - S_{LH}(t)] - [S_{HL}(t) - S_{HH}(t)] $$

(2)

where the survivor function, S(t), is the complement of the cumulative distribution function, F(t), and represents the probability that a response has not been made by time, t.

Different mental architectures also produce qualitatively distinct predictions for the SIC (Townsend & Nozawa, 1995; see Fig. 5) when using an exhaustive stopping rule (as is necessary for correct responses in the target category). Serial models predict an initially negative function which becomes positive, with the entire function integrating to zero (i.e., the MIC equals zero). Parallel models predict an entirely negative function. Coactive models predict an initial negative blip, with the majority of the SIC being positive. Coactive models do not integrate to zero but rather integrate to a positive value (note that since the target category uses an AND rule, all of the serial and parallel models, including those with a self-terminating rule, must predict an exhaustive SIC as shown in Fig. 5).^{Footnote 2}

Contrast category predictions

The contrast category stimuli can also be used to make diagnostic judgments using mean RTs (Fifić et al., 2010; see also Little, Eidels, Fific, & Wang, 2015, 2017). Illustrative predictions for each model are shown in Fig. 4. For example, consider the predictions of a fixed-order serial self-terminating model, in which dimension x is processed first followed by, if necessary, dimension y (for reference see Fig. 3, left panel). The presentation of items which satisfy the disjunctive contrast category rule on dimension x (i.e., x₀y₀, x₀y₁, x₀y₂) will lead to a decision without further processing. If, however, x₁y₀ or x₂y₀ are presented and x is processed first, then further processing of dimension y is necessary in order to make a correct categorization decision. This leads to a general prediction that RTs for the first processed dimension will be approximately equivalent, whereas RTs for the second processed dimension are comparatively slower (as they need to wait for the completion of the first processed dimension). Further, the exterior item (x₂y₀) is processed faster compared to the interior item (x₁y₀) because the first processed x dimension for the exterior stimulus is further from the decision boundary and therefore easier to judge as not belonging to the contrast category.

For mixed-order self-terminating models, the first-processed dimension may be dimension x on some trials and dimension y on other trials. It follows that the redundant stimulus has the greatest processing advantage, as both dimensions satisfy the disjunctive decision rule. Further, these models predict that exterior items will be processed faster than interior items on both dimensions. Averaged across trials, where participants switch from one dimension to the other, the time preceding the switch is shorter for exterior items.

For a parallel self-terminating model, the RT for a contrast category item is determined by the minimum processing time needed to make a categorization decision. More specifically, in the current task, the processing time will be determined by the dimension which yields a contrast category response. The redundant stimulus therefore has a processing advantage in this case as both dimensions yield contrast category responses, allowing for statistical facilitation between the dimensions (Raab, 1962).

In the case of exhaustive models, both dimensions are processed regardless of whether the disjunctive rule is satisfied or not. For a serial exhaustive model, this means that total RT comprises the sum of RTs on each single dimension. This leads to a prediction that the redundant stimulus will have the slowest RT (as it lies closest to both decision boundaries), with interior items being processed slower than exterior items. The parallel exhaustive model predicts that the total RT is determined by the maximum processing time needed to make a categorization decision. Again, redundant and interior items are slower as they lie closer to the decision boundaries compared to the exterior items.

Finally, a general prediction of the coactive architecture is that the interior stimuli will be processed faster than the exterior stimuli. The intuition is that the interior items are located closer to the left-most corner of the decision space, and therefore when the dimensions are pooled, interior items pool more evidence for a contrast category response compared to exterior items (where the evidence for one of the pooled dimensions is comparatively more diagnostic of a target category response).

This design offers considerable diagnosticity as data from each item can be used to differentiate architectures and associated stopping rules. As the analysis is non-parametric it does not rely on assumptions regarding the underlying probability distributions of the RTs. Further inferences are provided by formally instantiating each of the candidate architectures in a parametric RT model. In this model, the representation of each stimulus is instantiated as a multivariate signal detection model (e.g., using General Recognition Theory; Ashby & Townsend, 1986). This representation is used to derive rates of accumulation which are passed to a sequential sampling model (or pair of models for the serial and parallel models) to model the RT. These models, described fully below, allow us to account for correct and error RTs across all of the items simultaneously.

Current study

In the present study, we sought to determine the processing architecture underlying categorization decisions about luminance discs of different polarity in different spatial locations (e.g., lighter on the left, darker on the right). Requiring the discs to have different polarity ensured we would be able to manipulate the brightness of each disc independently without participants relying on the overall brightness of the display. However, due to the nature of the discriminability manipulation, it is possible that participants could make categorization decisions based on the overall contrast polarity (see Fig. 3); that is, the difference between the left and right disc luminance is greater for the target category stimuli compared to the contrast category stimuli. Consequently, a correct target category decision could be made based on high contrast polarity and a correct contrast category decision could be made based on a low contrast polarity. As we were interested in how participants made decisions based on individual luminance levels, rather than an overall evaluation of the contrast of the presented stimuli, we included a set of catch trial stimuli. These stimuli have high contrast polarity (now lighter on the right, darker on the left; essentially the reversal of the four target category items) but are associated with a contrast category response. This manipulation ensured participants could not make categorization decisions based on contrast, and instead necessitated that participants use the individual luminance levels of each disc.

We ran two versions of this experiment. In Experiment 1, the screen was divided by a series of boundary discs whose luminance levels were randomly sampled from the possible luminance levels of all discs in the category space. This was done to reduce the probability that items were grouped into a single object. These discs were removed in Experiment 2. Experiment 2, therefore, provides an almost direct replication of Experiment 1 but with a single minor methodological difference.

Experiments

General method

Participants

For Experiments 1 and 2, respectively, seven participants from the University of Melbourne community (14 in total, ten females and four males, aged between 19 and 26 years) with normal or corrected-to-normal vision completed the study. Participants were recruited via advertising placed on notice boards within the Melbourne School of Psychological Sciences and through the school’s online recruiting system. All participants were naïve to the purpose of the experiment. Participants provided informed consent and were reimbursed $10 per session, plus an extra $3 bonus for accuracy within a session greater than 90%. The participants from Experiment 1 are referred to as B1 - B7 (B denoting “boundary”), and participants from Experiment 2 are referred to as NB1 - NB7 (NB denoting “no boundary”). Testing was approved by the Melbourne Human Research Ethics Committee (Approval Number 1034866).

As we were interested in the individual-level decision mechanisms, we adopted an expert observer paradigm in which each observer acted as an independent replication of the experiment (see e.g., Little & Smith, 2018; Normand, 2016). Following relevant precedents, we collected a large number of trials for each individual item (N ≈ 300) in order to estimate the RT distribution for each item. As demonstrated in Smith and Little (2018), this approach has considerable advantages over traditional group designs which tend to be underpowered. Our goal is therefore not to estimate a population level parameter but is to test the predictions of each of the models.

Stimuli and apparatus

Illustrative examples of the stimuli used in Experiments 1 and 2 are shown in Fig. 3. Stimuli were presented at a monitor resolution of 1280 × 1024 and participants viewed the screen at a distance of approximately 60 cm. Stimuli were nine sets of two discs of different luminance levels presented on a gray background (RGB color space values [128 128 128]). The discs subtended a visual angle of 1.91^∘ with centers 11.34^∘ of visual angle to the left and right of fixation (the center of the screen).

For each stimulus, there was a white disc on the left and a black disc on the right. The specific level of luminance was varied. The set of stimuli was created by orthogonally combining the luminance level of the left disc and the luminance level of the right disc. These follow the logical rules design introduced by Fifić et al. (2010), whereby the discriminability manipulation was achieved by varying the luminance level of both discs by three possible increments in comparison to the background.^{Footnote 3} RGB coordinates were as follows for each salience manipulation: High salience black: [64 64 64], mid salience black: [88 88 88], low salience black: [112 112 112], high salience white: [200 200 200], mid salience white: [178 178 178], and low salience white: [156 156 156]. An additional four pairs of discs were added to act as catch trial stimuli. For these discs, the contrast was an orthogonal combination of high or mid salience black and white; however, the left disc was now darker than the background, and the right disc was lighter than the background (i.e., the contrasts were reversed compared to the primary experimental stimuli).

In Experiment 1, the screen was divided by a boundary of 29 discs (also subtended at a visual angle of 1.91^∘ at 60-cm viewing distance) presented as a central vertical column. The luminance values of the boundary discs were randomized from trial to trial using six possible RGB color space values (drawn from the values used to implement the salience manipulation). This boundary was removed in Experiment 2. All other aspects were the same. RTs for categorization were collected using a calibrated RT box (Li, Liang, Kleiner, & Lu, 2010).

Procedure

For Experiment 1, two participants completed five one-hour sessions of categorization on consecutive or near consecutive days.^{Footnote 4} The remaining participants from Experiments 1 and 2 completed six sessions. At the beginning of the task, participants were shown experimental instructions, as well as an example of the stimuli. Each session consisted of 867 trials (17 practice trials and 850 experimental trials). The contrast category stimuli (nine in total, including the catch trials) were presented five times per block, and the target category items (four in total) were presented ten times per block (i.e., 85 trials per block). This was done to minimize the development of a response bias for contrast category items. All stimuli were presented in randomized order within blocks. In between each block, participants were shown their percent correct on the current block and given the option to take a short break. During each trial, a fixation cross was presented for 1500 ms. A stimulus was then presented and participants were asked to decide whether the stimulus belonged to either category A or category B. Stimuli were presented for 5000 ms or until a response was made. Feedback was presented for incorrect responses. For responses greater than 5000 ms, the feedback “Too Slow” was presented and the trial was removed from the analysis. An example of a single trial for both experiments (with and without boundary discs) is shown in Fig. 6.

Data analysis

To analyze the target category, we focused on individual participant ANOVAs, using the interaction effect in a 2 × 2 factor design to assess the MIC. We used a series of planned t-tests to assess the pattern of contrast category RTs. These analyses follow relevant precedents (Fifić et al., 2010; Little et al., 2011, 2013), and allow us to make inferences about the specific processing patterns for each individual participant. Two shortcomings of this method are apparent. First, each analysis only considers a subset of the data. That is, even though the models make predictions across all of the items, an ANOVA across all nine items would be unwieldy and difficult to interpret. Second, each analysis only considers correct RTs. Although accuracy is high for most participants, a more complete analysis would also take into account patterns of error RTs. In order to deal with these issues, we complement our statistical analyses with computational model fitting in which we fit parametric instantiations of each of the models of interest (and relevant extensions) to the correct and error RT distributions for all of the items simultaneously. We then use model selection (i.e., the Deviance Information Criterion, DIC; Gelman, Hwang, & Vehtari, 2014) to select the model that provides the best explanation for our data. In summary, our analysis proceeds in two passes. We first use the non-parametric SFT analyses coupled with statistical tests to rule out specific models; we then instantiate the remaining models parametrically and fit them to the data comparing how well each fits the data taking into account the complexity of the model.

Experiment 1

Results

For all participants, the first session was considered practice and was excluded from further analysis. This was done to ensure participants had appropriately learned the categories and had developed a stable categorization strategy. Additionally, RTs less than 200 ms or greater than 3000 ms were excluded. These cut-offs are commonly used in the RT literature (see e.g., Donkin, Brown, & Heathcote, 2011). Less than 1% of trials in total were removed using this method. Mean correct RTs, mean error RTs, and error rates are presented in Table 1. Error rates tended to be low across all participants excepting in some cases for the LL, E_X, E_Y, and I_Y stimuli. While the analyses of SFT assume perfect accuracy (Townsend & Nozawa, 1995), Townsend and Wenger (2004) support the robustness of the SIC functions up to an error rate of 30%, which is much higher than that usually seen in studies utilizing the double factorial paradigm. The simulations of Fifić et al. (2008a) further demonstrate that estimates of the SIC are robust to violations of this assumption.

Table 1 Observed mean correct and error RTs (ms), and error rates for individual stimuli for each participant in Experiment 1

Full size table

In order to interpret the SIC functions, it is necessary that the Survivor functions are ordered such that S_HH(t) ≤ S_HL(t) ≈ S_LH(t) ≤ S_LL(t) with the strict inequality holding for at least one time point (Townsend & Nozawa, 1995). A series of Kolmogorov–Smirnov (KS) tests (Houpt, Blaha, McIntire, Havig, & Townsend, 2013) were used to check that each participant’s survivor functions followed this ordering. If the assumption of stochastic dominance holds, the first four columns of Table 11 should be significant, whereas the last four should not. No violations of stochastic dominance were found for this experiment (see Appendix A, Table 11). Survivor functions are shown in Appendix A, Fig. 14.

Target category

Figure 7 shows the mean RTs and corresponding MICs. All the MICs are near zero but in the positive direction.

To analyze the target category RTs, we conducted a series of 5 (sessions: 2-6) × 2 (left disc: L or H) × 2 (right disc: L or H) ANOVAs on the Target Category RTs for each individual participant (see Table 2).^{Footnote 5}

Table 2 Target category statistical results for individual participants in Experiment 1

Full size table

We first summarize the results which were common across all or most participants:

1.
There was a main effect of session, indicating that RTs become faster over the course of the experiment.
2.
There was a significant main effect of disc discriminability for both discs indicating that the discriminability manipulation was effective.
3.
For some participants, session interacted with one or both of the dimensions indicating for some sessions the left disc was processed faster than the right and vice versa.
4.
The three-way interaction was not significant, indicating a stable relationship between target category items across sessions. That is, participants were not changing processing strategy from session to session.
5.
With the exception of participant B6, the Left Disc × Right Disc interaction (see Fig. 7) was not significantly different from zero. Although the MIC was positive, a non-significant interaction is consistent with serial processing. The test of the interaction in the present case is a test of the point prediction of the serial model, which predicts that the interaction term should equal zero (cf. Sternberg, 1969). This presents a different goal to the typical null hypothesis significance testing case, where the goal of the significance cut-off is to place some criteria on the false-positive rate. In the present case, an alpha criterion of .05 is biased toward the serial model (Fox & Houpt, 2016). Consequently, caution must be taken when interpreting a non-significant result in this context.

Figure 8 shows the SICs. When considering the model predictions presented in Fig. 5, it can be seen that in all cases the SICs have a large positive portion, ruling out parallel processing for target category stimuli. Generally, the SICs also appear to have a greater positive region than negative region. This is most consistent with a coactive pattern of results.

Using two one-sided KS-Tests from Houpt’s (2013) SFT analysis package, we also sought to determine whether the positive and negative portions of the SICs were significantly different to zero. Two null-hypothesis tests were performed: one which determines whether the largest value of the SIC is significantly greater than zero (D+) and one which determines whether the lowest value of the SIC is significantly lower than zero (D-; see Houpt & Townsend, 2010). Like the MIC, the null hypothesis for the Houpt–Townsend statistic is SIC(t) = 0 for all times t, a conservative significance level biases the test toward retaining the null hypothesis (i.e., a serial model). We therefore adopted a less conservative cut-off of α = .33. This value has been shown to work well in model recovery tests using this statistic (Fox & Houpt, 2016). Both positive and negative D-tests are displayed in Table 3.

Table 3 Directional KS-tests for individual participants in Experiment 1

Full size table

For most participants, the positive deflection in the SIC is significantly greater than zero, whereas the negative deflection was not. This provides support for the coactive processing architecture. For B3, however, both positive and negative values were significant, which is indicative of either serial processing or coactive processing. For B5 neither value was significant.

Contrast category

The mean RTs for the contrast category are displayed in Fig. 9. For the majority of participants the interior stimulus was faster than the exterior stimulus, on at least one of the dimensions, which is suggestive of coactivity. No other model predicts a faster interior item compared to an exterior item on any dimension. Consequently, coactive processing provides a potential explanation for the contrast category items for most of the participants. Nonetheless, participant B1, despite having a faster interior compared to exterior item on one dimension, shows the reverse pattern on the other, which could be indicative of serial processing. For B4, the RTs seem more consistent with a fixed-order serial model, with the interior item being slower on one dimension, and the exterior and interior being approximately equal on the other. Further, for B5 both interior items were slower than exterior items, suggesting mixed-order serial self-terminating processing.

For contrast category items, we conducted a series of planned t tests comparing interior and exterior items on both dimensions and comparing the redundant stimulus to the other contrast category items (see Table 4). Except for two instances, the redundant stimulus was processed significantly faster than the other stimuli, providing evidence against an exhaustive stopping rule for the contrast category.^{Footnote 6}

Table 4 Contrast category statistical results for individual participants in Experiment 1

Full size table

Although a smaller mean RT was recorded for the interior items compared to the exterior items for the majority of participants, as expected under coactive processing, this pattern was only significant for B2 and B7 for the right dimension and B3 for the left dimension. Nonetheless, this pattern of faster interior than exterior items is not predicted by any other model. However, parallel processing cannot be ruled out since that model predicts no significant difference between the interior and exterior items. For B4 and B5, the exterior items were faster than the interior items, suggesting serial processing. However, again, this pattern was only significant for B5 on the right dimension.

Discussion

Taken together, the target category results for the majority of participants tend towards coactivity. While the MICs are positive for all participants except B5, the non-significant interaction between left and right dimensions in the target category is consistent with serial processing for all participants. When coupled with the SICs, however, a clearer pattern of coactivity emerges. First, the SICs all have a greater positive portion than negative portion which rules out parallel processing. Further, the positive portion of the SIC appears greater than the negative portion, suggesting coactive, rather than serial processing. This interpretation is supported by the directional KS-tests for most participants (excluding B3 and B5). Nonetheless, the target category data do not clearly rule-out serial processing.

Generally, the contrast category results also somewhat point to coactivity with the interior item being faster than the exterior item at least on one dimension for four of the seven participants. For three participants (B1, B4 and B5), an interior item was slower than an exterior item, which could indicate serial processing (although B1 also shows the opposite pattern on the other dimension, which is more indicative of coactive processing). We defer further discussion of these results until after the presentation of Experiment 2.