초록

본 연구의 목적은 학습상황에서 피드백으로 주어지는 금전적 획득/손실(학습 피드백)과 비학습적 상황에서 우연히 제시되는 의사 피드백(무선 피드백)을 비교하는 방법을 사용하여, 금전적 보상과 처벌의 학습 피드백으로서만 가지는 정보처리에 어느 두뇌 영역이 관여하는지를 규명하는 데 있다. 이를 위해 정상 성인(n = 22)을 대상으로 fMRI scan 동안 단서 자극에 대한 범주 버튼 반응(좌/우)의 정확 여부에 따라 피드백이 제시되는 시행(학습시행)과 단서 자극의 위치판단 반응과 무관하게 피드백이 제시되는 시행(무선시행)을 사건 관련 fMRI 방략으로 제시하였다. 두 시행 간 보상과 처벌과 같은 동기적 사건에 대한 두뇌 반응이 변별적으로 나타나는지를 알아보기 위해 시행 유형(학습 vs. 무선)과 피드백 유형(보상 vs, 처벌)을 두 독립변인으로 한 반복측정 이원분산분석을 하였다(voxel-wise FWE p < .001). 그 결과, 좌측 배외측 전두피질(dorsolateral prefrontal cortex), 좌측 전측 도(anterior insular), 배내측 전두피질(dorsomedial prefrontal cortex) 등의 영역에서 유의한 상호작용 효과가 관찰되었는데, 이들 영역은 모두 학습-보상 피드백 및 무선-처벌 피드백보다 학습-처벌 피드백에 대해 증가한 두뇌 활성을 보였다. 본 연구 결과는 학습상황에서 주어지는 처벌 피드백에 대한 기존 전략의 변경이나 재평가를 위한 집행적 처리, 적절하지 못하거나 틀린 행동에 대한 오류처리 과정 그리고 실패 경험에 대한 부정적 정서처리가 위에서 언급한 피질신경망을 중심으로 이루어질 가능성을 보여준다. 따라서 학습의 처벌 피드백은 보상과 달리 위와 같은 추가적 정보처리 과정이 존재할 가능성을 시사한다.

키워드

강화학습, 부적 피드백, 오류처리, 전측도, 전전두피질

참고문헌(73)open

  1. [학술지] Amiez, C. / 2006 / Reward encoding in the monkey anterior cingulate cortex / Cerebral Cortex 16 : 1040 ~ 1055

  2. [학술지] Anderson, A. K. / 2003 / Dissociated neural representations of intensity and valence in human olfaction / Nature Neuroscience 6 : 196 ~ 202

  3. [학술지] Aron, A. R. / 2004 / Human midbrain sensitivity to cognitive feedback and uncertainty during classification learning / Journal of Neurophysiology 92 : 1144 ~ 1152

  4. [학술지] Balleine, B. W. / 2007 / The role of the dorsal striatum in reward and decision-making / The Journal of Neuroscience 27 : 8161 ~ 8165

  5. [학술지] Bastin, J. / 2016 / Direct recordings from human anterior insula reveal its leading role within the error-monitoring network / Cerebral Cortex : bhv352

  6. [학술지] Berridge, K. C. / 1998 / What is the role of dopamine in reward : hedonic impact, reward learning, or incentive salience? / Brain Research Reviews 28 : 309 ~ 369

  7. [학술지] Bischoff-Grethe, A. / 2009 / The influence of feedback valence in associative learning / Neuroimage 44 : 243 ~ 251

  8. [학술지] Bray, S. / 2007 / Neural coding of reward-prediction error signals during classical conditioning with attractive faces / Journal of Neurophysiology 97 : 3036 ~ 3045

  9. [학술지] Breiter, H. C. / 2001 / Functional imaging of neural responses to expectancy and experience of monetary gains and losses / Neuron 30 : 619 ~ 639

  10. [학술지] Buckner, R. L. / 2008 / The brain’s default network / Annals of the New York Academy of Sciences 1124 : 1 ~ 38

  11. [학술지] Buzzell, G. A. / 2016 / Uncertainty-dependent activity within the ventral striatum predicts task-related changes in response strategy / Cognitive, Affective & Behavioral Neuroscience 16 : 219 ~ 233

  12. [학술지] Carreiras, M. / 2008 / Brain activation for consonants and vowels / Cerebral Cortex 18 : 1727 ~ 1735

  13. [학술지] Collins, A. G. / 2012 / How much of reinforcement learning is working memory, not reinforcement learning? A behavioral, computational, and neurogenetic analysis / European Journal of Neuroscience 35 : 1024 ~ 1035

  14. [학술지] Curtis, C. E. / 2003 / Persistent activity in the prefrontal cortex during working memory / Trends in Cognitive Sciences 7 : 415 ~ 423

  15. [학술지] Dale, A. M. / 1999 / Optimal experimental design for event-related fMRI / Human Brain Mapping 8 : 109 ~ 114

  16. [학술지] Debener, S. / 2005 / Trial-by-trial coupling of concurrent electroencephalogram and functional magnetic resonance imaging identifies the dynamics of performance monitoring / The Journal of Neuroscience 25 : 11730 ~ 11737

  17. [학술지] Delgado, M. R. / 2003 / Dorsal striatum responses to reward and punishment: effects of valence and magnitude manipulations / Cognitive, Affective & Behavioral Neuroscience 3 : 27 ~ 38

  18. [학술지] Delgado, M. R. / 2005 / An fMRI study of reward-related probability learning / Neuroimage 24 : 862 ~ 873

  19. [학술지] Fox, M. D. / 2005 / The human brain is intrinsically organized into dynamic, anticorrelated functional networks / Proceedings of the National Academy of Sciences 102 : 9673 ~ 9678

  20. [학술지] Gu, X. / 2013 / Anterior insular cortex and emotional awareness / The Journal of comparative neurology 521 : 3371 ~ 3388

  21. [학술지] Ham, T. / 2013 / Cognitive control and the salience network : an investigation of error processing and effective connectivity / The Journal of Neuroscience 33 : 7091 ~ 7098

  22. [학술지] Hare, T. A. / 2009 / Self-control in decision-making involves modulation of the vmPFC valuation system / Science 324 : 646 ~ 648

  23. [학술지] Holroyd, C. B. / 2002 / The neural basis of human error processing : reinforcement learning, dopamine, and the error-related negativity / Psychological Review 109 : 679 ~ 709

  24. [학술지] Holroyd, C. B. / 2004 / Dorsal anterior cingulate cortex shows fMRI response to internal and external error signals / Nature Neuroscience 7 : 497 ~ 498

  25. [학술지] Iannaccone, R. / 2015 / Conflict monitoring and error processing : New insights from simultaneous EEG–fMRI / Neuroimage 105 : 395 ~ 407

  26. [학술지] Kim, H. / 2006 / Is avoiding an aversive outcome rewarding? Neural substrates of avoidance learning in the human brain / PLoS Biology 4 : e233

  27. [학술지] 김수현 / 2015 / 학습의 진행과 두뇌 피드백 정보처리의 변화 / 한국심리학회지: 인지 및 생물 27 (3) : 419 ~ 450

  28. [학술지] Knutson, B. / 2001 / Anticipation of increasing monetary reward selectively recruits nucleus accumbens / The Journal of Neuroscience 21 : RC159

  29. [학술지] Knutson, B. / 2003 / A region of mesial prefrontal cortex tracks monetarily rewarding outcomes : characterization with rapid event-related fMRI / Neuroimage 18 : 263 ~ 272

  30. [학술지] Krawczyk, D. C. / 2002 / Contributions of the prefrontal cortex to the neural basis of human decision making / Neuroscience and Biobehavioral Reviews 26 : 631 ~ 664

  31. [학술지] Kuhnen, C. M. / 2005 / The neural basis of financial risk taking / Neuron 47 : 763 ~ 770

  32. [학술지] Liu, X. / 2011 / Common and distinct networks underlying reward valence and processing stages : a meta-analysis of functional neuroimaging studies / Neuroscience and Biobehavioral Reviews 35 : 1219 ~ 1236

  33. [학술지] Lotze, M. / 1999 / Activation of cortical and cerebellar motor areas during executed and imagined hand movements : an fMRI study / Journal of Cognitive Neuroscience 11 : 491 ~ 501

  34. [학술지] Luking, K. R. / 2013 / Candy and the brain: neural response to candy gains and losses / Cognitive, Affective & Behavioral Neuroscience 2013 (13) : 437 ~ 451

  35. [학술지] Maratos, E. J. / 2001 / Neural activity associated with episodic memory for emotional context / Neuropsychologia 39 : 910 ~ 920

  36. [학술지] Mckiernan, K. A. / 2003 / A parametric manipulation of factors affecting task-induced deactivation in functional neuroimaging / Journal of Cognitive Neuroscience 15 : 394 ~ 408

  37. [학술지] Menon, V. / 2010 / Saliency, switching, attention and control : a network model of insula function / Brain Structure & Function 214 : 655 ~ 667

  38. [학술지] Metereau, E. / 2013 / Cerebral correlates of salient prediction error for different rewards and punishments / Cerebral Cortex 23 : 477 ~ 487

  39. [학술지] Middleton, F. A. / 2002 / Basal-ganglia ‘projections’ to the prefrontal cortex of the primate / Cerebral Cortex 12 : 926 ~ 935

  40. [학술지] Montague, P. R. / 1996 / A framework for mesencephalic dopamine systems based on predictive Hebbian learning / The Journal of Neuroscience 16 : 1936 ~ 1947

  41. [학술지] O’Doherty, J. / 2003 / Dissociating valence of outcome from behavioral control in human orbital and ventral prefrontal cortices / The Journal of Neuroscience 23 : 7931 ~ 7939

  42. [학술지] O’Doherty, J. / 2004 / Dissociable roles of ventral and dorsal striatum in instrumental conditioning / Science 304 : 452 ~ 454

  43. [학술지] O’Doherty, J. / 2001 / Abstract reward and punishment representations in the human orbitofrontal cortex / Nature Neuroscience 4 : 95 ~ 102

  44. [학술지] Obleser, J. / 2010 / Segregation of Vowels and Consonants in Human Auditory Cortex : Evidence for Distributed Hierarchical Organization / Frontiers in Psychology 1 : 232

  45. [학술지] Pessiglione, M. / 2007 / How the brain translates money into force : a neuroimaging study of subliminal motivation / Science 316 : 904 ~ 906

  46. [학술지] Peterburs, J. / 2016 / The role of the human cerebellum in performance monitoring / Current Opinion in Neurobiology 40 : 38 ~ 44

  47. [학술지] Peters, S. / 2016 / Longitudinal development of frontoparietal activity during feedback learning : Contributions of age, performance, working memory and cortical thickness / Developmental Cognitive Neuroscience 19 : 211 ~ 222

  48. [학술지] Poldrack, R. A. / 2001 / Interactive memory systems in the human brain / Nature 414 : 546 ~ 550

  49. [학술지] Raichle, M. E. / 2001 / A default mode of brain function / Proceedings of the National Academy of Sciences 98 : 676 ~ 682

  50. [학술지] Ressler, N. / 2004 / Rewards and punishments, goal-directed behavior and consciousness / Neuroscience and Biobehavioral Reviews 28 : 27 ~ 39

  51. [학술지] Rustemeier, M. / 2016 / Processing of positive and negative feedback in patients with cerebellar lesions / Cerebellum 15 : 425 ~ 438

  52. [학술지] Salimpoor, V. N. / 2013 / Interactions between the nucleus accumbens and auditory cortices predict music reward value / Science 340 : 216 ~ 219

  53. [학술지] Samanez-Larkin, G. R. / 2008 / Individual differences in insular sensitivity during loss anticipation predict avoidance learning / Psychological Science 19 : 320 ~ 323

  54. [학술지] Schultz, W. / 1997 / Dopamine neurons and their role in reward mechanisms / Current Opinion in Neurobiology 7 : 191 ~ 197

  55. [학술지] Schultz, W. / 1997 / A neural substrate of prediction and reward / Science 275 : 1593 ~ 1599

  56. [학술지] Seger, C. A. / 2008 / How do the basal ganglia contribute to categorization? Their roles in generalization, response selection, and learning via feedback / Neuroscience and Biobehavioral Reviews 32 : 265 ~ 278

  57. [학술지] Seger, C. A. / 2005 / The roles of the caudate nucleus in human classification learning / The Journal of Neuroscience 25 : 2941 ~ 2951

  58. [학술지] Seger, C. A. / 2006 / Dynamics of frontal, striatal, and hippocampal systems during rule learning / Cerebral Cortex 16 : 1546 ~ 1555

  59. [학술지] Shackman, A. J. / 2011 / The integration of negative affect, pain and cognitive control in the cingulate cortex / Nature Reviews Neuroscience 12 : 154 ~ 167

  60. [학술지] Singer, T. / 2009 / A common role of insula in feelings, empathy and uncertainty / Trends in Cognitive Sciences 13 : 334 ~ 340

  61. [단행본] Skinner, B. F. / 1938 / The behavior of organisms: An experimental analysis / D. Appleton-Century Company

  62. [학술지] Spreng, R. N. / 2012 / The fallacy of a"task-negative"network / Frontiers in Psychology 3 : 145

  63. [학술지] Steele, V. R. / 2016 / Neuroimaging measures of error-processing : Extracting reliable signals from event-related potentials and functional magnetic resonance imaging / Neuroimage 132 : 247 ~ 260

  64. [학술지] Stephani, C. / 2011 / Functional neuroanatomy of the insular lobe / Brain Structure & Function 216 : 137 ~ 149

  65. [학술지] Stoodley, C. J. / 2012 / Functional topography of the cerebellum for motor and cognitive tasks : an fMRI study / Neuroimage 59 : 1560 ~ 1570

  66. [학술지] Taylor, S. F. / 1998 / The effect of emotional content on visual recognition memory : a PET activation study / Neuroimage 8 : 188 ~ 197

  67. [학술지] Tricomi, E. M. / 2004 / Modulation of caudate activity by action contingency / Neuron 41 : 281 ~ 292

  68. [학술지] Tricomi, E. M. / 2012 / Information content and reward processing in the human striatum during performance of a declarative memory task / Cognitive, Affective & Behavioral Neuroscience 12 : 361 ~ 372

  69. [학술지] Ullsperger, M. / 2003 / Error monitoring using external feedback : specific roles of the habenular complex, the reward system, and the cingulate motor area revealed by functional magnetic resonance imaging / The Journal of Neuroscience 23 : 4308 ~ 4314

  70. [학술지] Wheeler, E. Z. / 2008 / The human ventromedial frontal lobe is critical for learning from negative feedback / Brain 131 : 1323 ~ 1331

  71. [학술지] Williams, Z. M. / 2006 / Selective enhancement of associative learning by microstimulation of the anterior caudate / Nature Neuroscience 9 : 562 ~ 568

  72. [학술지] Xu, L. / 2009 / Neural mechanism of intertemporal choice : from discounting future gains to future losses / Brain Research 1261 : 65 ~ 74

  73. [학술지] Zanolie, K. / 2008 / Separable neural mechanisms contribute to feedback processing in a rule-learning task / Neuropsychologia 46 : 117 ~ 126