Evidence that instrumental conditioning requires conscious awareness in humans
Introduction
The extent to which learning can proceed without conscious awareness has been an extensively studied question. While there have been successful demonstrations that simple associations can be learned without conscious awareness of the stimuli, even between different modalities (e.g. Scott et al., 2018), the possibility of successful unconscious learning at increasing levels of complexity is unclear. One example of a complex learning process which has previously been suggested to proceed without cue awareness is instrumental conditioning, where agents learn the contingencies between stimuli in the world and their outcomes, and to selectively act upon them (e.g. approach/avoid; Mastropasqua and Turatto, 2015; Palminteri et al., 2012; Palminteri et al., 2009; Pessiglione et al., 2008). However, emerging evidence suggests that more complex forms of learning, including instrumental conditioning (Reber et al., 2018), contingency learning (Travers et al., 2018), and fear conditioning (Mertens and Engelhard, 2020) may not be possible unconsciously, as originally thought. Adding fuel to the debate are developments in experimental and statistical methods used to reliably assess absence of awareness (Dienes, 2015a; Rothkirch and Hesselmann, 2017; Shanks, 2017). With this in mind, we revisit the topic of unconscious instrumental conditioning. Instrumental learning is critical to adaptive behaviour (both ontogenetically and phylogenetically). As such, the extent to which it is feasible without conscious awareness is a question of considerable importance. In what follows, we attempt to conceptually replicate the original findings of Pessiglione et al. (2008), employing more sensitive methods and a more robust statistical approach, in two different paradigms: a trace conditioning task using monetary reinforcements, following the original task (Experiment 1), and a delay conditioning task with appetitive and aversive primary reinforcements (Experiment 2).
Past research into unconscious learning suggests that simple forms of associative learning, ranging from classical conditioning, to emotional, visuospatial, or multisensory learning can proceed without conscious awareness of the stimuli (Clark and Squire, 1998; Faivre et al., 2014; Knight et al., 2003; Lin and He, 2009; Olsson and Phelps, 2004; Rosenthal et al., 2016; Rosenthal et al., 2010; Scott et al., 2018; Seitz et al., 2009). Learning is usually indexed with presence or absence of the conditioned response, or with reaction times (e.g. where shorter reaction times to congruent vs incongruent pairings are indicative of learning). Unaware learning (as well as priming) is typically observed when stimuli are presented at short spatiotemporal intervals, or concurrently (van Gaal et al., 2012). For example, unaware classical conditioning was demonstrated in delay scenarios (where stimuli to be associated overlap temporally), but not in trace scenarios (where they are separated by an interval; Clark and Squire, 1998).
In contrast, instrumental conditioning can be classified as a more complex process. Agents must not only learn the associations between different stimuli, or stimuli and their outcomes, but also deploy action selectively (e.g. approach or avoid), and adapt their behaviour in the long-term. This involves integrating information over a long temporal scale and distinct modalities, as involved in processing the visual input, extracting its predictive value, deploying a selective response, processing the reinforcement, and comparing the expected outcome with the actual outcome in order to update the representations of stimulus value. As such, it is a considerably more complex process than the aforementioned simpler forms of associative learning, which do not require selective decisions on whether to act or not, or a behavioural adaptation from trial to trial.
Past theoretical and empirical accounts of conscious versus unconscious processing suggest that such an increased level of complexity should require conscious access. A number of theoretical contributions propose that consciousness is related to long-lasting, long-range connections between distinct brain regions, supporting recurrent information integration across distinct cognitive modules (Dehaene et al., 2014; Dehaene and Changeux, 2011; Dehaene and Naccache, 2001a; Lamme, 2006; Mudrik et al., 2014). As such, low-level or short-range (spatial or temporal) information integration might be possible without conscious awareness, but consciousness might be necessary at increased levels of complexity – for example, semantic knowledge, complex visual processing, decision-making and problem-solving (Baars, 2002), all of which involve integration of information across longer spatiotemporal intervals or larger spatial distance. Indeed, neuroimaging evidence has found conscious processing to be characterised by global, long-range spread of activity, in contrast to more localised, shorter-range projections when processing is unconscious (Baars, 2002; Baars et al., 2003; Dehaene et al., 2001; Dehaene et al., 2014; Dehaene and Naccache, 2001b; Melloni et al., 2007).
Another important consideration is related to the methodology of assessing absence of consciousness during task performance. Although it is a frequent practice in unconscious processing research to infer absence of awareness when a behavioural measure (e.g. conditioning, priming, etc.) is above chance, while an independent measure of awareness is not significantly different from chance (e.g. a separate unconscious discrimination task), this approach has drawn criticism (Dienes, 2015a; Vadillo et al., 2016). Finding that performance on a separate awareness check does not differ from chance is not enough to infer true absence of awareness – a non-significant result cannot disambiguate between absence of an effect (i.e. support for the null hypothesis) and absence of evidence for an effect (i.e. insensitive data). In a meta-analysis, Vadillo et al. (2016) argue that the seemingly chance-performance on this type of awareness test is more likely to reflect a false negative, especially in low-powered studies with a small sample. This approach, also adopted in the Pessiglione et al study, can be rectified in two ways. One is to ensure that the methods to assess awareness are relevant and sensitive (Berry and Dienes, 1993; Shanks, 2017; Shanks and St. John, 1994). This could be achieved by, for instance, a closer similarity between the awareness test and the measure of interest. The second is to apply statistical methods, such as the Bayes factor, which allows to determine whether a null result indicates support for the null (e.g. absence of awareness) over the alternative hypothesis (presence of awareness), or whether the data are insensitive (Dienes, 2014, Dienes, 2016; Sand and Nilsson, 2016). Both of those approaches will be used in the present paper to ensure true absence of awareness.
Instrumental conditioning is a crucial substrate of adaptive behaviour. Understanding whether it is achievable without conscious awareness is therefore of vital importance for theoretical and practical research investigating the boundaries of unconscious processing. With recent research casting doubt on feasibility of more complex forms of unconscious learning, as well as the methodological and statistical artifacts which can contribute to falsely assuming absence of awareness, it is imperative to rigorously address the question of feasibility of unconscious instrumental conditioning. Here, we present two experimental attempts to conceptually replicate Pessiglione and colleagues’ unconscious instrumental conditioning task (also employed in Mastropasqua and Turatto, 2015; Palminteri et al., 2009, and Palminteri et al., 2012). The original task constituted trace conditioning – participants learned the association between the subliminally presented stimulus and its outcome (presented supraliminally as monetary reinforcement), with an intervening gap of 3 seconds, during which action could have been executed (approach) or not (avoid). Experiment 1 is closely based on this paradigm, similarly adopting trace conditioning, and introducing minor methodological changes to enhance the sensitivity of the task. Experiment 2 seeks to evaluate whether unconscious instrumental conditioning can be achieved under conditions previously found to be more favourable for unconscious learning, namely where there is no temporal delay between the stimulus and outcome. Hence, Experiment 2 is a delay instrumental conditioning task, retaining the general structure of the original task, but ensuring that stimulus presentation overlaps with both the response, and the delivery of the reinforcement. To allow for the overlap of all three (which is not possible in just the visual domain), we replaced the secondary visual (monetary) reinforcement with primary reinforcement (appetitive and aversive tastes), which have been previously shown to be highly effective in achieving learning in humans (Birbaumer et al., 2015; Martin-Soelch et al., 2007; Parkinson et al., 2000). The strong spatiotemporal overlap was designed in order to maximise the chance of observing conditioning. Notably, primary reinforcements might also offer a stronger incentive to learn. In both experiments, we used Bayes Factors (including robustness regions) to determine whether the results sensitively support the null hypothesis (i.e. absence of learning) or the alternative hypothesis (presence of learning).
Section snippets
Experiment 1
Experiment 1 sought to conceptually replicate the unconscious instrumental conditioning task (Pessiglione et al., 2008), where participants learned to associate predictive cues with reward and punishment, and to adjust their behaviour accordingly by responding (Go) only to rewarding cues, and refraining from a response (NoGo) to punishing cues. Here, we retained the structure of the task, and introduced a number of changes to increase the sensitivity of the paradigm. Firstly, we used a
Experiment 2
Experiment 2 sought to conceptually replicate the unconscious instrumental conditioning task again, this time shortening the stimulus-outcome delay. To achieve this, we used a delay conditioning paradigm, while retaining as much of the structure of the original task as possible. As such, we employed the same Go/NoGo task, where participants learned to associate predictive cues with reward and punishment, and to adjust their behaviour accordingly by responding (Go) only to rewarding cues, and
General discussion
The extent to which complex forms of learning, such as instrumental conditioning, are possible without conscious awareness is still an unresolved question. To our knowledge, evidence for unconscious instrumental learning has been limited (Mastropasqua and Turatto, 2015; Pessiglione et al., 2008) and disputed (Reber et al., 2018). Yet, instrumental behaviour is a vital and one of the oldest forms of adaptive behaviour (both phylogenetically and ontogenetically). As such, we consider the extent
Credit Author Statement
LS: Conceptualisation, Investigation, Methodology, Formal analysis, Writing – original draft, review & editing. MY: Resources. HC: Resources. RS: Supervision, Methodology, Writing – review & editing.
Declaration of Competing Interest
None
Acknowledgments
This work was supported by the Sackler Centre for Consciousness Science, funded by the Sir Mortimer and Theresa Sackler Foundation, and the School of Psychology at the University of Sussex.
References (64)
The conscious access hypothesis: origins and recent evidence
Trends in Cognitive Sciences
(2002)Subjective experience is probably not limited to humans: The evidence from neurobiology and behavior
Consciousness and Cognition
(2005)- et al.
Brain, conscious experience and the observing self
Trends in Neurosciences
(2003) Operant conditioning in invertebrates
Current Opinion in NeurobiologyCurrent Opinion in Neurobiology
(2003)Spontaneous decisions and operant conditioning in fruit flies
Behavioural Processes
(2011)- et al.
Experimental and Theoretical Approaches to Conscious Processing
Neuron
(2011) - et al.
Towards a cognitive neuroscience of consciousness:Basic evidence and a workspace framework
Cognition
(2001) - et al.
Towards a cognitive neuroscience of consciousness: basic evidence and a workspace framework
Cognition
(2001) - et al.
Toward a computational theory of conscious processing
Current Opinion in Neurobiology
(2014) How Bayes factors change scientific practice
Journal of Mathematical Psychology
(2016)
Towards a true neural stance on consciousness
Trends in Cognitive Sciences
Seeing the invisible: The scope and limits of unconscious processing in binocular rivalry
Progress in Neurobiology
Appetitive conditioning: neural bases and implications for psychopathology
Neuroscience and Biobehavioral Reviews
The liabilities of mobility: A selection pressure for the transition to consciousness in animal evolution
Consciousness and Cognition
A systematic review and meta-analysis of the evidence for unaware fear conditioning
Neuroscience & Biobehavioral Reviews
Information integration without awareness
Trends in Cognitive Sciences
Similar improvement of reward and punishment learning by Serotonin Reuptake Inhibitors in Obsessive-Compulsive Disorder
Biological Psychiatry
Limbic cortical-ventral striatal systems underlying appetitive conditioning
Progress in Brain Research
Subliminal instrumental conditioning demonstrated in the human brain
Neuron
Cue discriminability predicts instrumental conditioning
Consciousness and Cognition
Learning and recognition of a non-conscious sequence of events inhuman primary visual cortex
Current Biology
Subliminal or not? Comparing null-hypothesis and Bayesian methods for testing subliminal priming
Consciousness and Cognition
Prevailing theories of consciousness are challenged by novel cross-modal associations acquired between subliminal stimuli
Cognition
Rewards evokelearning of unconsciously processed visual stimuli in adult humans
Neuron
Nonconscious learning from crowded sequences
Psychological Science
What insects can tell us about the origins of consciousness
Proceedings of the National Academy of Sciences
Implicit Learning: Theoretical and Empirical Issues
Appetitive vs. Aversive conditioning in humans. Front. Behav. Neurosci
The Psychophysics Toolbox
Spatial Vision
The operant and the classical in conditioned orientation of Drosophila melanogaster at the flight simulator
Learning and Memory
Surfing Uncertainty: Prediction, action and the embodied mind
Classical conditioning and brain systems: The role of awareness
Science
Cited by (34)
The role of consciousness in threat extinction learning
2023, Consciousness and CognitionThe effect of brief mindfulness training on the micro-structure of human free-operant responding: Mindfulness affects stimulus-driven responding; Mindfulness and schedule performance
2023, Journal of Behavior Therapy and Experimental PsychiatryPerceptual and semantic same-different processing under subliminal conditions
2023, Consciousness and CognitionFeasibility of unconscious instrumental conditioning: A registered replication
2023, CortexCitation Excerpt :As such, the threshold-setting task was amended to match the conditioning task more closely. In order to further enhance the sensitivity of the task and ensure that it reliably settles on sub-threshold conditions for the largest possible number of participants, stimulus contrast was reduced and the exposure time range was extended beyond the original limit of either 50 or 33 ms. Stimulus exposure was individually titrated for each participant in a stepwise manner based on both the accuracy and reported confidence in visual perceptions.2 This procedure was used extensively in previous research for effective identification of sub-threshold conditions (Scott et al., 2018; Skora, Yeomans, Crombag, & Scott, 2021; Skora, Livermore, Nisini, & Scott, 2022). Secondly, the separate awareness check was replaced with a trial-by-trial measure, allowing to access the information about participants’ awareness in a more immediate fashion (Berry & Dienes, 1993; Newell & Shanks, 2014).
Can insects feel pain? A review of the neural and behavioural evidence
2022, Advances in Insect PhysiologyCitation Excerpt :Although not a compelling demonstration of sentience on its own, such learning goes over and above reflex-like responses to noxious stimuli. There is ongoing debate about which forms of associative learning are linked to sentience and conscious awareness (Ginsburg and Jablonka, 2019) with particularly intense recent interest in certain forms of instrumental conditioning (Skora et al., 2021), reversal learning (Travers et al., 2018), and trace conditioning (Droege et al., 2021). To leave room for this debate, without taking a stance, we review both classical conditioning and more complex forms of learning, which may provide stronger evidence for sentience.