Abstract
Public policy research typically neglects the role of the individual policy actor with most accounts of the policy process instead privileging the role of governmental systems, institutions, processes, organizations; organised interests or networks of multiple actors. The policy design literature suffers from similar limitations, with very few authors paying attention to the crucial work of the individual policy designer or considering how the latter’s skills, expertise and creativity are employed in the design task. This represents a significant weakness in our understanding of how policy is formulated. This paper outlines and previews what we believe is a potentially fruitful semi-experimental methodological tool for exploring how individual policy actors draw on knowledge, expertise, intuition and creativity in framing and responding to complex policy issues. Real-time scenario-based problem-solving exercises are used to explore how policy problems and solutions are framed and articulated by novice (first-term politicians and early career bureaucrats) and experienced (former cabinet ministers and senior civil servants) policy actors and to examine the strategies and approaches they employ in response to specific problem cues. Initial findings are discussed, and we conclude by advancing potential refinements of the instrument and directions for future research.
Similar content being viewed by others
Notes
This constitutes an initial sample which will be extended to approximately 70–80 respondents by the end of the project. The sample frame includes all ex-state and federal ministers who had served at least two parliamentary terms; senior bureaucrats included all ex department secretaries who had served at least 10 years in a senior leadership position at the state and/or federal level; first-term backbenchers included all federal and Victorian parliamentary members serving their first term in office; early career policy officers included state and federal public officials in the first 5 years of a policy role. Recruitment for ex-ministers, ex-senior bureaucrats and backbenchers was by direct invitation with all those responding favourably to interview. Early career policy officers were recruited by invitation through a professional association, the Institute for Public Administration Australia (IPAA) Victorian branch. This sample is currently heavily skewed as it includes no female ministers and only a small number of senior bureaucrats. The final sample will reflect a quota sampling methodology with approximately equal numbers across all four subpopulations and a proportional representation across genders.
The research design incorporates questionnaire-based scale items (administered face-to-face); ‘emblematic’ case study analysis and a series of scenario-based problem-solving exercises.
For the purposes of this article, coding and assessment of responses were carried out by one of the authors. Once the full sample is collected, responses will be blind-coded and categorised by a panel of two researchers independent of the research team.
Prior to addressing the scenario, Jackson had outlined his involvement and approach in two other policy issues.
This suggests an important contextual variable which needs to be considered in any analysis of variations in expertise levels displayed by novice and experienced policy-makers—the impact of domain-specific experience either in terms of jurisdiction, portfolio responsibilities, or prior experience. For example, it is possible that a first-term backbencher with a background in policing or the criminal justice system may display superior levels of domain-specific expertise in responding to this scenario than an ex federal minister with years of experience in a completely unrelated port-folio area. To control for this and to identify cases where prior experience may aid the development of domain-specific expertise, we are collecting detailed biographical information on all participants. We are also using two diverse scenario-based exercises in each interview as a way of ensuring at least one response will be drawn from an unfamiliar domain.
See the work of the Delft Centre for Serious Gaming for a leading example of developments in this area. www.seriousgaming.tudelft.nl.
References
Alexander, E. R. (1979). The design of alternatives in organizational contexts: A pilot study. Administrative Science Quarterly (ASQ), 42(3), 382–404.
Alexander, E. R. (1982). Design in the decision making process. Policy Sciences, 14(3), 279–292.
Alexander, D., Considine, M., & Lewis, J. M. (2013). Solving problems in policy and management: The role of skills, expertise and experience. In Conference paper presented at XVII IRSPM conference, Prague, April 10–12, 2013.
Allain, C., & Sarrazin, C. (1990). Study of decision-making in squash competition: A computer simulation approach. Canadian Journal of Sport Science, 15(3), 193–200.
Allison, G. T. (1971). Essence of decision: Explaining the Cuban missile crisis. Boston: Little Brown.
Bakema, W. E., & Secker, I. P. (1988). Ministerial expertise and the Dutch case. European Journal of Political Research, 16(2), 153–170.
Beckman, L. (2006). The competent Cabinet? Ministers in Sweden and the problem of competence and democracy. Scandanavian Political Studies, 26(2), 111–129.
Borgeaud, P., & Abernethy, B. (1987). Skilled perception in volleyball defense. Journal of Sport Psychology, 9, 400–406.
Bozeman, B., & Scott, P. (1992). Laboratory experiments in public policy and management. Journal of Public Administration, Research and Theory, 2(3), 293–313.
Chabal, P. M. (2003). Do ministers matter? The individual style of ministers in programmed policy change. International Review of Administrative Sciences, 69(1), 29–49.
Considine, M. (2005). Making public policy: Institutions, actors, strategies. Cambridge: Polity Press.
Considine, M. (2012). Thinking outside the box: Applying design theory to public policy. Politics and Policy, 40(4), 704–724.
Considine, M., Lewis, J. M., & Alexander, D. (2009). Networks, innovation and public policy: Politicians, bureaucrats and the pathways to change inside government. Houndmills, Basinstoke: Palgrave Macmillan.
Dorst, K. (1995). ‘Editorial’. Special issue on: Analysing design activity: New directions in protocol analysis. Design Studies, 16, 139–142.
Dryzek, J. S. (1983). Don’t toss coins in garbage cans: A prologue to policy design. Journal of Public Policy, 3, 345–367.
Furnham, A., & Boo, H. C. (2011). A literature review of the anchoring effect. The Journal of Socioeconomics, 40, 35–42.
Greenstein, F. (1969). Personality and politics: Problems of evidence, inference and conceptualisation. Chicago: Markham Pub Co.
Guetzkow, H., Alger, C., Brody, R., Noel, R. & Snyder, R. (1963). Simulation in international relations: developments for research and teaching. Englewood Cliffs, NJ: Prentice-Hall, Inc.
Hajer, M. A. (1995). The politics of environmental discourse: Ecological modernization and the policy process. Oxford: Oxford University Press.
Headey, B. W. (1974). The role skills of Cabinet ministers: A cross-national review. Political Studies, 22, 66–85.
Hood, C., & Margetts, H. (2007). The tools of government in the digital age. Houndmills, Basinstoke: Palgrave Macmillan.
Howlett, M., & Lejano, R. P. (2012). Tales from the Crypt: The rise and fall (and Rebirth?) of policy design. Administration and Society, XX(X), 1–25.
Hudson, V. M. (2005). Foreign policy analysis: Actor-specific theory and the ground of international relations. Foreign Policy Analysis, 1(1), 1–30.
Jervis, R. (1988). Realism, game theory, and cooperation. World Politics, 40(3), 317–349.
Jiang, H., & Yen, C. C. (2009). Protocol analysis in design research: A review. Paper presented at the “Design Rigor & Relevance”, International association of societies of design research (IASDR) 2009 Conference, Seoul, Korea.
Joldersma, C., Geurts, J. L., Vermaas, J., & Heyne, G. (1995). A policy exercise for the Dutch health care system for the elderly. In D. Crookall & K. Arai (Eds.), Simulation and gaming across disciplines and cultures: ISAGA at a watershed (pp. 111–121). Thousand Oaks, CA: Sage Publications.
Kahan, J. P., Rydell, C. P., & Setear, J. (1995). A game of urban drug policy. Peace and Conflict: Journal of Peace Psychology, 1(3), 275–290, (RAND Reprint RP-459).
Kahneman, D., Slovic, P., & Tversky, A. (1982). Judgment under uncertainty: Heuristics and biases. Cambridge: Cambridge University Press.
Kingdon, J. (1995). Agendas, alternatives and public policies (2nd ed.). Boston: Harper Collins.
Kneebone, R. (2003). Simulation in surgical training: Educational issues and practical applications. Medical Education, 37, 267–277.
Kuhnert, K. W. (2001). Leadership theory in postmodernist organizations. In R. T. Golembiewski (Ed.), The nature of organizational leadership. San Francisco: Jossey-Bass.
Law, B., Atkins, M., Kirkpatrick, A., Lomax, A., & Mackenzie, C. (2004). Eye gaze patterns differentiate novice and experts in a MIST VR laparoscopic simulator. Studies in Health Technology and Informatics, 50, 124–130.
Linder, S. H., & Peters, B. G. (1984). From social theory top policy design. Journal of Public Policy, 4, 237–249.
Marris, C., Joly, P., Ronda, S., & Bonneuil, C. (2003). Precautionary expertise for GM crops (PEG). National Workshop Report, France.
Mascarenhas, D. R. D., Collins, D., Mortimer, P., & Morris, R. L. (2005). A naturalistic approach to training accurate and coherent decision making in rugby union referees. The Sport Psychologist, 19, 131–147.
May, P. J. (1981). Hints for crafting alternative policies. Policy Analysis, 7, 227–244.
Mayer, I. (2009). The gaming of policy and the politics of gaming: A review. Simulation and Gaming, 40(6), 825–862.
Mintrom, M. (2000). Policy entrepreneurs and school choice. Washington, DC: Georgetown University Press.
Mintrom, M., & Norman, P. (2009). Policy entrepreneurship and policy change. The Policy Studies Journal, 37(4), 649–667.
Norman, E., Brooks, L., & Hamstra, S. (2006). Expertise in medicine and surgery. In E. Ericsson, N. Charness, P. J. Feltovich, & R. Hoffman (Eds.), Cambridge handbook of expertise and expert performance (pp. 339–353). Cambridge: Cambridge University Press.
Olshfski, D., & Cunningham, R. (2008). Agendas and decisions how state government executives and middle managers make and administer policy. Albany: State University of New York Press.
Oreszczyn, S., & Carr, S. (2008). Improving the link between policy research and practice: Using a scenario workshop as a qualitative research tool in the case of genetically modified crops. Qualitative Research, 8(4), 473–497.
Parson, E. (1996). A Global Climate-Change Policy Exercise: Results of a Test Run, July 27–29, 1995.
Polsby, N. W. (1984). Political innovation in America: The politics of policy initiation. New Haven, Conn: Yale University Press.
Ranyard, R., & Williamson, J. (2005). Conversation-based process tracing methods for the study of naturalistic decision making: Analysing information search and verbal protocols. In H. Montgomery, R. Lipshitz, & B. Brehmer (Eds.), How professionals make decisions. Mahwah, NJ: LEA Associates.
Rhodes, R. A. W. (2005). Everyday life in a ministry: Public administration as anthropology. American Review of Public Administration, 20(1), 1–23.
Rhodes, R. A. W. (2011). Everyday life in British government. Oxford: Oxford University Press.
Rhodes, R. A. W., ‘t Hart, P., & Noordegraaf, M. (Eds.). (2007). Observing government elites: Up close and personal. Basingstoke: Palgrave Macmillan.
Rhodes, R. A. W., & Weller, P. (Eds.). (2001). The changing world of top officials: Mandarins or valets?. Buckingham, UK: Open University Press.
Ross, K. G., Shaffer, J. L., & Klein, G. (2006). Professional judgments and “naturalistic decision making”. In K. A. Ericsson, N. Charness, R. R. Hoffman & P. J. Feltovich (Eds.), The Cambridge handbook of expertise and expert performance. New York: Cambridge University Press.
Schafer, M., & Walker, S. G. (2006). Democratic leaders and the democratic peace: The operational codes of Tony Blair and Bill Clinton. International Studies Quarterly, 50(3), 561–583.
Schaph, F. (1991). Games real actors could play. The challenge of complexity. Journal of Theoretical Politics, 3(3), 277–304.
Schaph, F. (1994). Games real actors could play. Positive and negative coordination in embedded negotiations. Journal of Theoretical Politics, 6(1), 27–53.
Schneider, A., & Ingram, H. (1988). Systematically pinching ideas: A comparative approach to policy design. Journal of Public Policy, 8(1), 61–80.
Schon, D. (1983). The reflective practitioner: How professionals think in action. London: Temple Smith.
Shannon, V. P., & Keller, J. W. (2007). Leadership style and international norm violation: The case of the Iraq war. Foreign Policy Analysis, 3(1), 79–104.
Simon, H. A. (1978). Rational decision making in business organizations. Nobel Memorial Lecture.
Simon, H. A. (1981). The sciences of the artificial. Cambridge, MA: The MIT Press.
Smith, R. (2010). The long history of gaming in military training. Simulation and Gaming: An Interdisciplinary Journal, 41(1), 6–19.
Theakston, K. (2006). What makes for an effective British prime minister? Paper prepared for the 2006 annual meeting of the American Political Science Association.
Tiernan, A., & Weller, P. (2010). Learning to be a minister: Heroic expectations, practical realities. Melbourne: Melbourne University Press.
Torkington J, Smith S.G., Rees B.I., & Darzi, A. (2001). ‘Skill transfer from virtual reality to a real laparoscopic task’, Surgical Endoscopy, 15(10), 1076–1079
Toth, F. L. (1988a). Policy exercises: Objectives and design elements. Simulation and Gaming: An Interdisciplinary Journal, 19, 235–255.
Toth, F. L. (1988b). Policy exercises: Procedures and implementation. Simulation and Gaming: An Interdisciplinary Journal, 19, 256–276.
Toth, F., & Hizsnyik, E. (2004). Managing the inconceivable: Participatory assessments of impacts and responses to extreme climate change. Laxenburg, Austria: International Institute for Applied Systems Analysis.
Tsebelis, G. (1990). Nested games: Rational choice in comparative politics. California: University of California Press.
Tversky, A., & Kahneman, D. (1974). Judgement under uncertainty: Heuristics and biases. Science, 185(4157), 1124–1131.
Valkenburg, R., & Dorst, K. (1998). The reflective practice of design teams. Design Studies, 19(3), 249–271.
Walls, J., Bertrand, L., Gale, T., & Saunders, N. (1998). Assessment of upwind dinghy sailing performance using a virtual reality dinghy sailing simulator. Journal of Science and Medicine in Sport, 1(2), 61–72.
Ward, P., Williams, A. M., & Bennett, S. J. (2002). Visual search and biological motion perception in tennis. Research Quarterly for Exercise and Sport, 73, 107–112.
Ward, P., Williams, M., & Hancock, P. (2006). Simulation for performance and training. In E. Ericsson, N. Charness, P. J. Feltovich, & R. Hoffman (Eds.), Cambridge handbook of expertise and expert performance (pp. 243–262). Cambridge: Cambridge University Press.
Weimer, D. (1993). ’The current state of design craft: Borrowing, tinkering, and problem solving. Public Administration Review, 53(2), 110–120.
Weimer, D. (1998). Policy analysis and evidence: A craft perspective. Policy Studies Journal, 26(1), 114–128.
Weller, P., & Grattan, M. (1981). Can ministers cope? Australian federal ministers at work. Richmond, Vic: Hutchinson.
Williams, A. M., & Burwitz, L. (1993). Advance cue utilization in soccer. In T. Reilly, J. Clarys, & A. Stibbe (Eds.), Science and football II (pp. 239–244). London: E. & F. N. Spon.
Williamson, J., Raynard, R., & Cuthbert, L. (2000). A conversation-based process tracing method for use with naturalistic decisions: An evaluation study. British Journal of Psychology, 91, 203–221.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Considine, M., Alexander, D. & Lewis, J.M. Policy design as craft: teasing out policy design expertise using a semi-experimental approach. Policy Sci 47, 209–225 (2014). https://doi.org/10.1007/s11077-013-9191-0
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11077-013-9191-0