Abstract
The major purpose of this paper is to propose a comprehensive model describing the effects of response sets within the theory framework of the stages of responding to questionnaires, and taking into account the effects of collectivist and individualist attributes within cross-cultural contexts. The introduction of this model aims to provide a construct that may help minimize biases in questionnaire-based research as well as providing new directions for theoretical and empirical research in the field of response sets.
Similar content being viewed by others
References
Andersen S.M., Reznik I., Chen S. (1997) The self in relation to others: Cognitive and motivational underpinnings. In: Snodgrass J.G., Thompson R.L. (eds). The self across psychology. Academy of Sciences, New York, pp. 233–275
Anderson N.H. (1981) Foundations of information integration theory. Academic Press, New York
Arce-Ferrer A.J. (2006) An Investigation Into the Factors Influencing Extreme-Response Style. Educational and Psychological Measurement 66(3): 374–392
Arnold H.J., Feldman D.C. (1981) Social desirability response bias in self-report choice situations. Academy of Management Journal 24(2): 377–385
Bachman J.G., O’Malley P.M. (1984a) Black-white differences in self-esteem: Are they affected by response styles?. American Journal of Sociology 90: 624–639
Bachman J.G., O’Malley P.M. (1984b) Yea-saying, nay-saying, and going to extremes: Black-white differences in response styles. Public Opinion Quarterly 48(2): 491–509
Barnette J. (2000) Effects of stem and Likert response option reversals on survey internal consistency: If you feel the need, there is a better alternative to using those negatively worded stems. Educational and Psychological Measurement 60(3): 361–370
Beatty P., Herrmann D. (2002) To answer or not to answer: Decision process related to survey item nonresponse. In: Groves R.N., Dillman D.A., Eltinge J.L., Little R.J. (eds). Survey nonresponse. John Wiley & Sons, New York, pp. 71–85
Bellah R., Madsen R., Sullivan W., Swidler A., Tipton S. (1985). Habits of the heart: Individualism and commitment in American life. University of California Press, Berkeley
Billiet J.B., McClendon M.J. (2000) Modeling acquiescence in measurement models for two balanced sets of items. Structural Equation Modeling 7(4): 608–628
Bishop G.F., Smith A. (2001) Response order effects and the early Gallop split ballots. Public Opinion Quarterly 65: 479–505
Bishop G.F., Tuchfarber A.J., Oldendick R.W. (1986) Opinion fictitious issues: The pressure to answer survey questions. Public Opinion Quarterly 50: 240–250
Bless H., Igou E.R., Schwartz N., Waenke M. (2000) Reducing context effects by adding context information: The direction and size of context effects in political judgment. Personality and Social Psychology Bulletin 26(9): 1036–1045
Bradburn N.M., Sudman S., Blair E., Stocking C. (1978) Question threat and response bias. Public Opinion Quarterly 42(2): 221–234
Brew F.P., Hesketh B., Taylor A. (2001) Individualist-collectivist differences in adolescent decision making and decision styles with Chinese and Anglos. International Journal of Intercultural Relations 25(1): 1–19
Buchanan T., Ali T., Heffernan T.M., Ling J., Parrott A.C., Rodgers J. et al. (2005) Nonequivalence of on-line and paper-and-pencil psychological tests: The case of the prospective memory questionnaire. Behavior Research Methods 37(1): 148–154
Cantril H. (1946) The intensity of an attitude. Journal of Abnormal and Social Psychology 41: 129–135
Chen C., Lee S.-Y., Stevenson H.W. (1995) Response style and cross-cultural comparisons of rating scales among East Asian and North American students. Psychological Science 6(3): 170–175
Chiou J.-S. (2001) Horizontal and vertical individualism and collectivism among college students in the United States, Taiwan, and Argentina. Journal of Social Psychology 141(5): 667–678
Clarke I., III (2000) Extreme response style in cross-cultural research: An empirical investigation. Journal of Social Behavior and Personality 15(1): 137–152
Couch A., Keniston K. (1960) Yeasayers and naysayers: Agreeing response set as a personality variable. Journal of Abnormal and Social Psychology 60: 151–174
Crandall J.E. (1965) Some relationships among sex, anxiety, and conservatism of judgment. Journal of Personality 33(1):99–107
Cronbach L.J. (1946) Response sets and test validity. Educational and Psychological Measurement 6: 475–494
Crowne D.P., Marlowe D. (1960) A new scale of social desirability independent of psychopathology. Journal of Consulting Psychology 24: 349–354
Edwards A.L. (1957) The social desirability in personality assessment and research. Holt, Rinehart & Winstone, New York
Edwards A.L. (1963) A factor analysis of experimental social desirability and response set scales. Journal of Applied Psychology 47(5): 308–316
Edwards A.L. (1966) Relationship between probability of endorsement and social desirability scale value for a set of 2,824 personality statements. Journal of Applied Psychology 50(3): 238–239
Edwards A.L., Diers C. (1963) Neutral items as a measure of acquiescence. Educational and Psychological Measurement 23(4): 687–698
Edwards A.L., Walker J.N. (1961) Social desirability and agreement response set. Journal of Abnormal and Social Psychology 62: 180–183
Fishbein M., Ajzen I. (1981) On construct validity: A critique of Miniard and Cohen’s paper. Journal of Experimental Social Psychology 17(3): 340–350
Fiske A.P. (1992) The four elementary forms of sociality: Framework for a unified theory of social relations. Psychological Review 99(4): 689–723
Gilljam M., Granberg D. (1993) Should we take don’t know for an answer?. Public Opinion Quarterly 57(3): 348–357
Greenleaf E.A. (1992a) Improving rating scale measures by detecting and correcting bias components in some response styles. Journal of Marketing Research 29(2): 176–188
Greenleaf E.A. (1992b) Measuring extreme response style. Public Opinion Quarterly 56(3): 328–351
Grimm S.D., Church A. (1999) A cross-cultural study of response biases in personality measures. Journal of Research in Personality 33(4): 415–441
Groves R.M. (1989) Survey errors and survey costs. John Wiley & Sons, New York
Gudykunst W.B. (1997) Cultural variability in communication: An introduction. Communication Research 24(4): 327–348
Gudykunst W.B., Matsumoto Y. (1996) Cross-cultural variability of communication in personal relationships. In: Gudykunst W.B., Ting-Toomey S., Nishida T. (eds). Communication in personal relationships across cultures. Sage, Thousand Oaks
Gudykunst W.B., Matsumoto Y., Ting-Toomey S., Nishida T. (1996) The influence of cultural individualism-collectivism, self construals, and individual values on communication styles across cultures. Human Communication Research 22(4): 510–543
Haberstroh S., Oyserman D., Schwarz N., Kuehnen U., Ji L.-J. (2002) Is the interdependent self more sensitive to question context than the independent self? Self-construal and the observation of conversational norms. Journal of Experimental Social Psychology 38(3): 323–329
Harzing A.-W. (2006) Response Styles in Cross-national Survey Research: A 26-country Study. International Journal of Cross Cultural Management 6(2): 243–266
Heine S.J., Lehman D.R. (1995) Social desirability among Canadian and Japanese students. Journal of Social Psychology 135(6): 777–779
Heine S.J., Lehman D.R. (1997) The cultural construction of self-enhancement: An examination of group-serving biases. Journal of Personality and Social Psychology 72(6): 1268–1283
Heine S.J., Lehman D.R., Peng K., Greenholtz J. (2002) What’s wrong with cross-cultural comparisons of subjective Likert scales?: The reference-group effect. Journal of Personality andSocial Psychology 82(6): 903–918
Hofstede G. (1980) Culture’s consequences: International differences in work-related values. Sage, Beverly Hills
Hofstede G. (1991) Cultures and organizations: Software of the mind. McGraw-Hill, London
Holtgraves T. (1997) Styles of language use: Individual and cultural variability in conversational indirectness. Journal of Personality and Social Psychology 73(3): 624–637
Holtgraves T. (2004) Social desirability and self-reports: Testing models of socially desirable responding. Personality & Social Psychology Bulletin 30(2): 161–172
Hsu F.L. (1983) Rugged individualism reconsidered. University of Tennessee Press, Knoxville
Hui C. (1988) Measurement of individualism-collectivism. Journal of Research in Personality 22(1): 17–36
Hui C., Triandis H. (1989) Effects of culture and response format on extreme response style. Journal of Cross-Cultural Psychology 20(3): 296–309
Javeline D. (1999) Response effects in polite cultures: A test of acquiescence in Kazakhstan. Public Opinion Quarterly 63(1): 1–28
Johnson J. (1981) Effects of the order of presentation of evaluative dimensions for bipolar scales in four societies. Journal of Social Psychology 113(1): 21–27
Johnson T., Kulesa P., Cho Y.I., Shavitt S. (2005) The relation between culture and response styles: Evidence from 19 Countries. Journal of Cross-Cultural Psychology 36(2): 264–277
Johnson T.P., O’Rourke D., Chavez N., Sudman S., Warnecke R.B., Lacey L. et al. (1997) Social cognition and responses to survey questions among culturally diverse populations. In: Lyberg L., Biemer P., Collin M., de Leeuw E., Dippo C., Schwarz N., Trewin D. (eds). Survey measurement and process quality. Wiley-Interscience, New York
Kagitcibasi, C. (1994). A critical appraisal of individualism and collectivism: Toward a new formulation. In U. Kim, H. C. Triandis, C. Kagitcibasi, S.-C. Choi, & G. Yoon (Eds.), Individualism and collectivism: Theory, method, and applications (Vol. 18, pp. 52–65). Newbury Park: Sage.
Kim U., Triandis H., Kagitcibasi C., Choi S.-C., Yoon G., eds. (1994) Individualism and collectivism: Theory, method, and applications. Sage, Thousand Oaks, CA
Knowles E.S., Condon C.A. (1999) Why people say “yes”: A dual-process theory of acquiescence. Journal of Personality and Social Psychology 77(2): 379–386
Knowles E.S., Nathan K.T. (1997) Acquiescent responding in self-reports: Cognitive style or social concern?. Journal of Research in Personality 31(2): 293–301
Krosnick J.A. (1991) Response strategies for coping with the cognitive demands of attitude measures in surveys. Applied Cognitive Psychology 5: 213–236
Krosnick J.A. (2002) The cause of no-opinion response to attitude measures in surveys: They are rarely what they appear to be. In: Groves R.N., Dillman D.A., Eltinge J.L., Little R.J. (eds). Survey nonresponse. John Wiley & Sons, New York
Krosnick J.A., Holbrook A.L., Berent M.K., Carson R.T., Hanemann W., Kopp R.J., et al. (2002) The impact of “no opinion” response options on data quality: Non-attitude reduction or an invitation to satisfice?. Public Opinion Quarterly 66(3): 371–403
Krosnick J.A., Schuman H. (1988) Attitude intensity, importance, and certainty and susceptibility to response effects. Journal of Personality and Social Psychology 54(6): 940–952
Kuehnen U., Oyserman D. (2002) Thinking about the self influences thinking in general: Cognitive consequences of salient self-concept. Journal of Experimental Social Psychology 38(5): 492–499
Lee C., Green R.T. (1991) Cross cultural examination of Fishbein behavioral intentions model. Journal of International Business Studies 22: 289–305
Lehnert W. (1977) Human and computational question answering. Cognitive Science 1(1): 47–73
Marin G., Gamba R.J., Marin B.V. (1992) Extreme response style and acquiescence among Hispanics: The role of acculturation and education. Journal of Cross-Cultural Psychology 23(4): 498–509
Markus H.R., Kitayama S. (1991) Culture and the self: Implications for cognition, emotion, and motivation. Psychological Review 98(2): 224–253
Mathiowetz N.A., Duncan G.J. (1988) Out of work, out of mind: Response errors in retrospective reports of unemployment. Journal of Business and Economic Statistics 6: 221–229
Matsuda Y., Harsel S., Furusawa S., Kim H.-S., Quarles J. (2001) Democratic values and mutual perceptions of human rights in four Pacific Rim nations. International Journal of Intercultural Relations 25(4): 405–421
McClendon M.J. (1991) Acquiescence and recency response-order effects in interview surveys. Sociological Methods and Research 20: 60–103
Middleton K.L., Jones J.L. (2000) Socially desirable response sets: The impact of country culture. Psychology and Marketing 17(2): 149–163
Mondak J.J., Davis M.B. (2001) Asked and Answered: Knowledge Levels When We Will Not Take Don’t Know for an Answer. Political Behavior 23(3): 199–224
Moore D.W. (2002) Measuring new types of question-order effects: Additive and subtractive. Public Opinion Quarterly 66(1): 80–91
Moorman R.H., Podsakoff P.M. (1992) A meta-analytic review and empirical test of the potential confounding effects of social desirability response sets in organizational behaviour research. Journal of Occupational and Organizational Psychology 65(2): 131–149
Morling B., Fiske S.T. (1999) Defining and measuring harmony control. Journal of Research in Personality 33(4): 379–414
Norenzayan A., Schwarz N. (1999) Telling what they want to know: Participants tailor causal attributions to researchers’ interests. European Journal of Social Psychology 29(8): 1011–1020
Ohbuchi K.-I., Fukushima O., Tedeschi J.T. (1999) Cultural values in conflict management: Goal orientation, goal attainment, and tactical decision. Journal of Cross-Cultural Psychology 30(1): 51–71
Oppenheim A.N. (1966) Questionnaire design and attitude measurement. Heinemann, London
Oyserman D. (1993) The lens of personhood: Viewing the self and others in a multicultural society. Journal of Personality and Social Psychology 65(5): 993–1009
Oyserman D., Coon H.M., Kemmelmeier M. (2002) Rethinking individualism and collectivism: Evaluation of theoretical assumptions and meta-analyses. Psychological Bulletin 128(1): 3–72
Oyserman, D., & Markus, H. R. (1993). The sociocultural self. In J. M. Suls (Ed.), The self in social perspective (Vol. 4, pp. 187–220). Hillsdale, NJ: Lawrence Erlbaum Associates.
Paulhus, D. L. (1991). Measurement and control of response bias. In J. P. Robinson, P. R. Shaver, L. S. Wrightsman, & F. M. Andrews (Eds.), Measures of personality and social psychological attitudes (Vol. 1, pp. 17–59). San Diego: Academic Press.
Paulhus D.L. (2002) Socially desirable responding: The evolution of a construct. In: Braun H.I., Jackson D.N., Wiley D.E., Messick S. (eds). The role of constructs in psychological and educational measurement. Lawrence Erlbaum Associates, Mahwah, NJ, pp. 49–69
Paulhus D.L., Harms P.D., Bruce M.N., Lysy D.C. (2003) The over-claiming technique: Measuring self-enhancement independent of ability. Journal of Personality and Social Psychology 84(4): 890–904
Paulhus D.L., John O.P. (1998) Egoistic and moralistic biases in self-perception: The interplay of self-deceptive styles with basic traits and motives. Journal of Personality 66(6): 1025–1060
Paulhus D.L., Reid D.B. (1991) Enhancement and denial in socially desirable responding. Journal of Personality and Social Psychology 60(2): 307–317
Peterson J.B., DeYoung C.G., Driver-Linn E., Seguin J.R., Higgins D.M., Arseneault L., et al. (2003) Self-deception and failure to modulate responses despite accruing evidence of error. Journal of Research in Personality 37(3): 205–223
Ray J. (1983) Reviving the problem of acquiescent response bias. Journal of Social Psychology 121(1): 81–96
Reykowski J. (1994) Collectivism and individualism as dimensions of social change. In: Kim U., Triandis H.C., Kagitcibasi C., Choi C., Yoon G. (eds). Individualism and collectivism: Theory, method, and applications. Thousand Oaks, California
Richman W.L., Kiesler S., Weisband S., Drasgow F. (1999) A meta-analytic study of social desirability distortion in computer-administered questionnaires, traditional questionnaires, and interviews. Journal of Applied Psychology 84(5): 754–775
Sampson E.E. (1977) Psychology and the American ideal. Journal of Personality and Social Psychology 35(11): 767–782
Schuman H., Presser S. (1981) Questions and answers in attitude surveys: Experiments on question form, wording, and context. Academic Press, New York
Schwartz S.H. (1990) Individualism-collectivism: Critique and proposed refinements. Journal of Cross-Cultural Psychology 21(2): 139–157
Schwarz N. (1999) Self-reports: How the questions shape the answers. American Psychologist 54(2): 93–105
Schwarz N. (2003) Self-reports in consumer research: The challenge of comparing cohorts and cultures. Journal of Consumer Research 29(4): 588–594
Schwarz N., Hippler H.-J. (1995) Subsequent questions may influence answers to preceding questions in mail surveys. Public Opinion Quarterly 59(1): 93–97
Schwarz N., Hippler H.-J., Deutsch B., Strack F. (1985) Response scales: Effects of category range on reported behavior and comparative judgments. Public Opinion Quarterly 49(3): 388–395
Schwarz N., Hippler H.J., Noelle-Neumann E. (1991) Cognitive model of response-order effects. In: Schwarz N., Sudman S. (eds). Context effects in social and psychological research. Springer Verlag, New York
Schwarz N., Oyserman D. (2001) Asking questions about behavior: Cognition, communication, and questionnaire construction. American Journal of Evaluation 22(2): 127–160
Sekaran U. (1984) Methodological and theoretical issues and advancements in cross-cultural research. Journal of International Business Studies 14(2): 61–73
Shulruf, B., Hattie, J., & Dixon, R. (2006). The influence of individualist and collectivist attributes on responses to Likert-type scales. Paper presented at the 26th International Association of Applied Psychology, 17–21 July, Athens.
Shulruf, B., Hattie, J., & Dixon, R. (2007). Development of a New Measurement Tool for Individualism and Collectivism. Journal of Psychoeducational Assessment (in press).
Shulruf, B., Watkins, D., Hattie, J., Faria, L., Pepi, A., Alesi, M., et al. (in progress). Measuring Collectivism and Individualism in the Third Millennium.
Singelis T.M., Triandis H., Bhawuk D., Gelfand M.J. (1995) Horizontal and vertical dimensions of individualism and collectivism: A theoretical and measurement refinement. Cross-Cultural Research: The Journal of Comparative Social Science 29(3): 240–275
Smith P.B. (2004) Acquiescent Response Bias as an Aspect of Cultural Communication Style. Journal of Cross-Cultural Psychology 35(1): 50–61
Stening B., Everett J. (1984) Response styles in a cross-cultural managerial study. Journal of Social Psychology 122(2): 151–156
Strack F. (1992) Order effects in survey research: Activation and information functions of preceding questions. In: Schwarz N., Sudman S. (eds). Context effects in social and psychological research. Springer-Verlag, New York, pp. 23–34
Sudman S., Bradburn N.M. (1974) Response effects in surveys. Aldine Publishing Company, Chicago
Sudman S., Bradburn N.M., Schwarz N. (1996) Thinking about answers: The application of cognitive processes to survey methodology. Jossey-Bass, San Francisco
Swearingen, D. L. (1998). Response sets, item format, and thinking style: Implications for questionnaire design. U Denver, US, 1.
Tourangeau R. (1991) Context effects on responses to attitude questions: Attitudes as memory structure. In: Schwarz N., Sudman S. (eds). Context effects in social and psychological research. Springer-Verlag, New York, pp. 35–47
Tourangeau R. (2003) Cognitive aspects of survey measurement and mismeasurement. International Journal of Public Opinion Research 15(1): 3–7
Tourangeau R., Rasinski K.A. (1988) Cognitive processes underlying context effects in attitude measurement. Psychological Bulletin 103(3): 299–314
Tourangeau R., Smith T.W. (1996) Asking sensitive questions: The impact of data collection mode, question format, and question context. Public Opinion Quarterly 60(2): 275–304
Triandis H. (1989) The self and social behavior in differing cultural contexts. Psychological Review 96(3): 506–520
Triandis H. (1995) Individualism and collectivism. Westview Press, Boulder
Triandis H. (1996) The psychological measurement of cultural syndromes. American Psychologist 51(4): 407–415
Triandis H., Bontempo R., Villareal M.J., Asai M., Lucca N. (1988) Individualism and collectivism: Cross-cultural perspectives on self in group relationships. Journal of Personality and Social Psychology 54(2): 323–338
Triandis H., Gelfand M. (1998) Converging measurement of horizontal and vertical individualism and collectivism. Journal of Personality and Social Psychology 74(1): 118–128
Triandis H., McCusker C., Hui C. (1990) Multimethod probes of individualism and collectivism. Journal of Personality and Social Psychology 59(5): 1006–1020
Triandis H., Suh E.M. (2002) Cultural influences on personality. Annual Review of Psychology 53(1): 133–160
Uskul A.K., Oyserman D. (2005) Question Comprehension and Response: Implications of Individualism and Collectivism. In: Mannix B., Neale M., Chen Y. (eds). Research on managing groups and teams: National culture & groups. Elsevier Science, Oxford
van Herk H., Poortinga Y.H., Verhallen T.M. (2004) Response styles in rating scales: Evidence of method bias in data from six EU countries. Journal of Cross-Cultural Psychology 35(3): 346–360
Walsh W.A., Banaji M.R., eds. (1997) The collective self (Vol. 818). Annals of the New York Academy of Sciences, New York
Warnecke R.B., Johnson T.P., Chavez N., Sudman S., O’Rourke D., Lacey L., et al. (1997) Improving question wording in survey of culturally diverse population. Annual Epidemiology 7: 334–342
Waterman A.S. (1984) The psychology of individualism. Praeger, New York
Watson D. (1992) Correcting for acquiescent response bias in the absence of a balanced scale: An application to class consciousness. Sociological Methods and Research 21(1): 52–88
Welkenhuysen-Gybels J., Billiet J., Cambre B. (2003) Adjustment for acquiescence in the assessment of the construct equivalence of Likert-type score items. Journal of Cross-Cultural Psychology 34(6): 702–722
Weng L.-J., Cheng C.-P. (2000) Effects of response order on Likert-type scales. Educational and Psychological Measurement 60(6): 908–924
Wilson T.D., LaFleur S.J., Anderson D. (1996) The validity and consequences of verbal reports about attitudes. In: Schwarz N., Sudman S. (eds). Answering questions: Methodology for determining cognitive and communicative processes in survey research. Jossey-Bass, San Francisco, pp. 91–114
Wong N., Rindfleisch A., Burroughs J. (2003) Do Reverse-Worded Items Confound Measures in Cross-Cultural Consumer Research? The Case of the Material Values Scale. Journal of Consumer Research 30: 72–91
Wu S., Rubin D.L. (2000) Evaluating the impact of collectivism and individualism on argumentative writing by Chinese and North American college students. Research in Teaching in English 35: 148–178
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Shulruf, B., Hattie, J. & Dixon, R. Factors affecting responses to Likert type questionnaires: introduction of the ImpExp, a new comprehensive model. Soc Psychol Educ 11, 59–78 (2008). https://doi.org/10.1007/s11218-007-9035-x
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11218-007-9035-x