Skip to main content
Log in

Policy design as craft: teasing out policy design expertise using a semi-experimental approach

  • Published:
Policy Sciences Aims and scope Submit manuscript

Abstract

Public policy research typically neglects the role of the individual policy actor with most accounts of the policy process instead privileging the role of governmental systems, institutions, processes, organizations; organised interests or networks of multiple actors. The policy design literature suffers from similar limitations, with very few authors paying attention to the crucial work of the individual policy designer or considering how the latter’s skills, expertise and creativity are employed in the design task. This represents a significant weakness in our understanding of how policy is formulated. This paper outlines and previews what we believe is a potentially fruitful semi-experimental methodological tool for exploring how individual policy actors draw on knowledge, expertise, intuition and creativity in framing and responding to complex policy issues. Real-time scenario-based problem-solving exercises are used to explore how policy problems and solutions are framed and articulated by novice (first-term politicians and early career bureaucrats) and experienced (former cabinet ministers and senior civil servants) policy actors and to examine the strategies and approaches they employ in response to specific problem cues. Initial findings are discussed, and we conclude by advancing potential refinements of the instrument and directions for future research.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

Notes

  1. This constitutes an initial sample which will be extended to approximately 70–80 respondents by the end of the project. The sample frame includes all ex-state and federal ministers who had served at least two parliamentary terms; senior bureaucrats included all ex department secretaries who had served at least 10 years in a senior leadership position at the state and/or federal level; first-term backbenchers included all federal and Victorian parliamentary members serving their first term in office; early career policy officers included state and federal public officials in the first 5 years of a policy role. Recruitment for ex-ministers, ex-senior bureaucrats and backbenchers was by direct invitation with all those responding favourably to interview. Early career policy officers were recruited by invitation through a professional association, the Institute for Public Administration Australia (IPAA) Victorian branch. This sample is currently heavily skewed as it includes no female ministers and only a small number of senior bureaucrats. The final sample will reflect a quota sampling methodology with approximately equal numbers across all four subpopulations and a proportional representation across genders.

  2. The research design incorporates questionnaire-based scale items (administered face-to-face); ‘emblematic’ case study analysis and a series of scenario-based problem-solving exercises.

  3. For the purposes of this article, coding and assessment of responses were carried out by one of the authors. Once the full sample is collected, responses will be blind-coded and categorised by a panel of two researchers independent of the research team.

  4. Prior to addressing the scenario, Jackson had outlined his involvement and approach in two other policy issues.

  5. This suggests an important contextual variable which needs to be considered in any analysis of variations in expertise levels displayed by novice and experienced policy-makers—the impact of domain-specific experience either in terms of jurisdiction, portfolio responsibilities, or prior experience. For example, it is possible that a first-term backbencher with a background in policing or the criminal justice system may display superior levels of domain-specific expertise in responding to this scenario than an ex federal minister with years of experience in a completely unrelated port-folio area. To control for this and to identify cases where prior experience may aid the development of domain-specific expertise, we are collecting detailed biographical information on all participants. We are also using two diverse scenario-based exercises in each interview as a way of ensuring at least one response will be drawn from an unfamiliar domain.

  6. See the work of the Delft Centre for Serious Gaming for a leading example of developments in this area. www.seriousgaming.tudelft.nl.

References

  • Alexander, E. R. (1979). The design of alternatives in organizational contexts: A pilot study. Administrative Science Quarterly (ASQ), 42(3), 382–404.

    Article  Google Scholar 

  • Alexander, E. R. (1982). Design in the decision making process. Policy Sciences, 14(3), 279–292.

    Article  Google Scholar 

  • Alexander, D., Considine, M., & Lewis, J. M. (2013). Solving problems in policy and management: The role of skills, expertise and experience. In Conference paper presented at XVII IRSPM conference, Prague, April 10–12, 2013.

  • Allain, C., & Sarrazin, C. (1990). Study of decision-making in squash competition: A computer simulation approach. Canadian Journal of Sport Science, 15(3), 193–200.

    Google Scholar 

  • Allison, G. T. (1971). Essence of decision: Explaining the Cuban missile crisis. Boston: Little Brown.

    Google Scholar 

  • Bakema, W. E., & Secker, I. P. (1988). Ministerial expertise and the Dutch case. European Journal of Political Research, 16(2), 153–170.

    Article  Google Scholar 

  • Beckman, L. (2006). The competent Cabinet? Ministers in Sweden and the problem of competence and democracy. Scandanavian Political Studies, 26(2), 111–129.

    Article  Google Scholar 

  • Borgeaud, P., & Abernethy, B. (1987). Skilled perception in volleyball defense. Journal of Sport Psychology, 9, 400–406.

    Google Scholar 

  • Bozeman, B., & Scott, P. (1992). Laboratory experiments in public policy and management. Journal of Public Administration, Research and Theory, 2(3), 293–313.

    Google Scholar 

  • Chabal, P. M. (2003). Do ministers matter? The individual style of ministers in programmed policy change. International Review of Administrative Sciences, 69(1), 29–49.

    Google Scholar 

  • Considine, M. (2005). Making public policy: Institutions, actors, strategies. Cambridge: Polity Press.

    Google Scholar 

  • Considine, M. (2012). Thinking outside the box: Applying design theory to public policy. Politics and Policy, 40(4), 704–724.

    Google Scholar 

  • Considine, M., Lewis, J. M., & Alexander, D. (2009). Networks, innovation and public policy: Politicians, bureaucrats and the pathways to change inside government. Houndmills, Basinstoke: Palgrave Macmillan.

    Book  Google Scholar 

  • Dorst, K. (1995). ‘Editorial’. Special issue on: Analysing design activity: New directions in protocol analysis. Design Studies, 16, 139–142.

    Article  Google Scholar 

  • Dryzek, J. S. (1983). Don’t toss coins in garbage cans: A prologue to policy design. Journal of Public Policy, 3, 345–367.

    Article  Google Scholar 

  • Furnham, A., & Boo, H. C. (2011). A literature review of the anchoring effect. The Journal of Socioeconomics, 40, 35–42.

    Google Scholar 

  • Greenstein, F. (1969). Personality and politics: Problems of evidence, inference and conceptualisation. Chicago: Markham Pub Co.

    Google Scholar 

  • Guetzkow, H., Alger, C., Brody, R., Noel, R. & Snyder, R. (1963). Simulation in international relations: developments for research and teaching. Englewood Cliffs, NJ: Prentice-Hall, Inc.

  • Hajer, M. A. (1995). The politics of environmental discourse: Ecological modernization and the policy process. Oxford: Oxford University Press.

    Google Scholar 

  • Headey, B. W. (1974). The role skills of Cabinet ministers: A cross-national review. Political Studies, 22, 66–85.

    Article  Google Scholar 

  • Hood, C., & Margetts, H. (2007). The tools of government in the digital age. Houndmills, Basinstoke: Palgrave Macmillan.

    Google Scholar 

  • Howlett, M., & Lejano, R. P. (2012). Tales from the Crypt: The rise and fall (and Rebirth?) of policy design. Administration and Society, XX(X), 1–25.

    Google Scholar 

  • Hudson, V. M. (2005). Foreign policy analysis: Actor-specific theory and the ground of international relations. Foreign Policy Analysis, 1(1), 1–30.

    Article  Google Scholar 

  • Jervis, R. (1988). Realism, game theory, and cooperation. World Politics, 40(3), 317–349.

    Article  Google Scholar 

  • Jiang, H., & Yen, C. C. (2009). Protocol analysis in design research: A review. Paper presented at the “Design Rigor & Relevance”, International association of societies of design research (IASDR) 2009 Conference, Seoul, Korea.

  • Joldersma, C., Geurts, J. L., Vermaas, J., & Heyne, G. (1995). A policy exercise for the Dutch health care system for the elderly. In D. Crookall & K. Arai (Eds.), Simulation and gaming across disciplines and cultures: ISAGA at a watershed (pp. 111–121). Thousand Oaks, CA: Sage Publications.

    Google Scholar 

  • Kahan, J. P., Rydell, C. P., & Setear, J. (1995). A game of urban drug policy. Peace and Conflict: Journal of Peace Psychology, 1(3), 275–290, (RAND Reprint RP-459).

    Google Scholar 

  • Kahneman, D., Slovic, P., & Tversky, A. (1982). Judgment under uncertainty: Heuristics and biases. Cambridge: Cambridge University Press.

    Book  Google Scholar 

  • Kingdon, J. (1995). Agendas, alternatives and public policies (2nd ed.). Boston: Harper Collins.

    Google Scholar 

  • Kneebone, R. (2003). Simulation in surgical training: Educational issues and practical applications. Medical Education, 37, 267–277.

    Article  Google Scholar 

  • Kuhnert, K. W. (2001). Leadership theory in postmodernist organizations. In R. T. Golembiewski (Ed.), The nature of organizational leadership. San Francisco: Jossey-Bass.

  • Law, B., Atkins, M., Kirkpatrick, A., Lomax, A., & Mackenzie, C. (2004). Eye gaze patterns differentiate novice and experts in a MIST VR laparoscopic simulator. Studies in Health Technology and Informatics, 50, 124–130.

    Google Scholar 

  • Linder, S. H., & Peters, B. G. (1984). From social theory top policy design. Journal of Public Policy, 4, 237–249.

    Article  Google Scholar 

  • Marris, C., Joly, P., Ronda, S., & Bonneuil, C. (2003). Precautionary expertise for GM crops (PEG). National Workshop Report, France.

  • Mascarenhas, D. R. D., Collins, D., Mortimer, P., & Morris, R. L. (2005). A naturalistic approach to training accurate and coherent decision making in rugby union referees. The Sport Psychologist, 19, 131–147.

    Google Scholar 

  • May, P. J. (1981). Hints for crafting alternative policies. Policy Analysis, 7, 227–244.

    Google Scholar 

  • Mayer, I. (2009). The gaming of policy and the politics of gaming: A review. Simulation and Gaming, 40(6), 825–862.

    Article  Google Scholar 

  • Mintrom, M. (2000). Policy entrepreneurs and school choice. Washington, DC: Georgetown University Press.

    Google Scholar 

  • Mintrom, M., & Norman, P. (2009). Policy entrepreneurship and policy change. The Policy Studies Journal, 37(4), 649–667.

    Article  Google Scholar 

  • Norman, E., Brooks, L., & Hamstra, S. (2006). Expertise in medicine and surgery. In E. Ericsson, N. Charness, P. J. Feltovich, & R. Hoffman (Eds.), Cambridge handbook of expertise and expert performance (pp. 339–353). Cambridge: Cambridge University Press.

    Chapter  Google Scholar 

  • Olshfski, D., & Cunningham, R. (2008). Agendas and decisions how state government executives and middle managers make and administer policy. Albany: State University of New York Press.

    Google Scholar 

  • Oreszczyn, S., & Carr, S. (2008). Improving the link between policy research and practice: Using a scenario workshop as a qualitative research tool in the case of genetically modified crops. Qualitative Research, 8(4), 473–497.

    Article  Google Scholar 

  • Parson, E. (1996). A Global Climate-Change Policy Exercise: Results of a Test Run, July 27–29, 1995.

  • Polsby, N. W. (1984). Political innovation in America: The politics of policy initiation. New Haven, Conn: Yale University Press.

    Google Scholar 

  • Ranyard, R., & Williamson, J. (2005). Conversation-based process tracing methods for the study of naturalistic decision making: Analysing information search and verbal protocols. In H. Montgomery, R. Lipshitz, & B. Brehmer (Eds.), How professionals make decisions. Mahwah, NJ: LEA Associates.

    Google Scholar 

  • Rhodes, R. A. W. (2005). Everyday life in a ministry: Public administration as anthropology. American Review of Public Administration, 20(1), 1–23.

    Google Scholar 

  • Rhodes, R. A. W. (2011). Everyday life in British government. Oxford: Oxford University Press.

    Google Scholar 

  • Rhodes, R. A. W., ‘t Hart, P., & Noordegraaf, M. (Eds.). (2007). Observing government elites: Up close and personal. Basingstoke: Palgrave Macmillan.

    Google Scholar 

  • Rhodes, R. A. W., & Weller, P. (Eds.). (2001). The changing world of top officials: Mandarins or valets?. Buckingham, UK: Open University Press.

    Google Scholar 

  • Ross, K. G., Shaffer, J. L., & Klein, G. (2006). Professional judgments and “naturalistic decision making”. In K. A. Ericsson, N. Charness, R. R. Hoffman & P. J. Feltovich (Eds.), The Cambridge handbook of expertise and expert performance. New York: Cambridge University Press.

  • Schafer, M., & Walker, S. G. (2006). Democratic leaders and the democratic peace: The operational codes of Tony Blair and Bill Clinton. International Studies Quarterly, 50(3), 561–583.

    Article  Google Scholar 

  • Schaph, F. (1991). Games real actors could play. The challenge of complexity. Journal of Theoretical Politics, 3(3), 277–304.

    Article  Google Scholar 

  • Schaph, F. (1994). Games real actors could play. Positive and negative coordination in embedded negotiations. Journal of Theoretical Politics, 6(1), 27–53.

    Article  Google Scholar 

  • Schneider, A., & Ingram, H. (1988). Systematically pinching ideas: A comparative approach to policy design. Journal of Public Policy, 8(1), 61–80.

    Article  Google Scholar 

  • Schon, D. (1983). The reflective practitioner: How professionals think in action. London: Temple Smith.

    Google Scholar 

  • Shannon, V. P., & Keller, J. W. (2007). Leadership style and international norm violation: The case of the Iraq war. Foreign Policy Analysis, 3(1), 79–104.

    Article  Google Scholar 

  • Simon, H. A. (1978). Rational decision making in business organizations. Nobel Memorial Lecture.

  • Simon, H. A. (1981). The sciences of the artificial. Cambridge, MA: The MIT Press.

    Google Scholar 

  • Smith, R. (2010). The long history of gaming in military training. Simulation and Gaming: An Interdisciplinary Journal, 41(1), 6–19.

    Article  Google Scholar 

  • Theakston, K. (2006). What makes for an effective British prime minister? Paper prepared for the 2006 annual meeting of the American Political Science Association.

  • Tiernan, A., & Weller, P. (2010). Learning to be a minister: Heroic expectations, practical realities. Melbourne: Melbourne University Press.

    Google Scholar 

  • Torkington J, Smith S.G., Rees B.I., & Darzi, A. (2001). ‘Skill transfer from virtual reality to a real laparoscopic task’, Surgical Endoscopy, 15(10), 1076–1079

    Google Scholar 

  • Toth, F. L. (1988a). Policy exercises: Objectives and design elements. Simulation and Gaming: An Interdisciplinary Journal, 19, 235–255.

    Article  Google Scholar 

  • Toth, F. L. (1988b). Policy exercises: Procedures and implementation. Simulation and Gaming: An Interdisciplinary Journal, 19, 256–276.

    Article  Google Scholar 

  • Toth, F., & Hizsnyik, E. (2004). Managing the inconceivable: Participatory assessments of impacts and responses to extreme climate change. Laxenburg, Austria: International Institute for Applied Systems Analysis.

    Google Scholar 

  • Tsebelis, G. (1990). Nested games: Rational choice in comparative politics. California: University of California Press.

    Google Scholar 

  • Tversky, A., & Kahneman, D. (1974). Judgement under uncertainty: Heuristics and biases. Science, 185(4157), 1124–1131.

    Article  Google Scholar 

  • Valkenburg, R., & Dorst, K. (1998). The reflective practice of design teams. Design Studies, 19(3), 249–271.

    Article  Google Scholar 

  • Walls, J., Bertrand, L., Gale, T., & Saunders, N. (1998). Assessment of upwind dinghy sailing performance using a virtual reality dinghy sailing simulator. Journal of Science and Medicine in Sport, 1(2), 61–72.

    Article  Google Scholar 

  • Ward, P., Williams, A. M., & Bennett, S. J. (2002). Visual search and biological motion perception in tennis. Research Quarterly for Exercise and Sport, 73, 107–112.

    Article  Google Scholar 

  • Ward, P., Williams, M., & Hancock, P. (2006). Simulation for performance and training. In E. Ericsson, N. Charness, P. J. Feltovich, & R. Hoffman (Eds.), Cambridge handbook of expertise and expert performance (pp. 243–262). Cambridge: Cambridge University Press.

    Chapter  Google Scholar 

  • Weimer, D. (1993). ’The current state of design craft: Borrowing, tinkering, and problem solving. Public Administration Review, 53(2), 110–120.

    Article  Google Scholar 

  • Weimer, D. (1998). Policy analysis and evidence: A craft perspective. Policy Studies Journal, 26(1), 114–128.

    Article  Google Scholar 

  • Weller, P., & Grattan, M. (1981). Can ministers cope? Australian federal ministers at work. Richmond, Vic: Hutchinson.

    Google Scholar 

  • Williams, A. M., & Burwitz, L. (1993). Advance cue utilization in soccer. In T. Reilly, J. Clarys, & A. Stibbe (Eds.), Science and football II (pp. 239–244). London: E. & F. N. Spon.

    Google Scholar 

  • Williamson, J., Raynard, R., & Cuthbert, L. (2000). A conversation-based process tracing method for use with naturalistic decisions: An evaluation study. British Journal of Psychology, 91, 203–221.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Mark Considine.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Considine, M., Alexander, D. & Lewis, J.M. Policy design as craft: teasing out policy design expertise using a semi-experimental approach. Policy Sci 47, 209–225 (2014). https://doi.org/10.1007/s11077-013-9191-0

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11077-013-9191-0

Keywords

Navigation