The impact of outdoor youth programs on positive adolescent development: Study protocol for a controlled crossover trial

https://doi.org/10.1016/j.ijer.2017.10.004Get rights and content

Highlights

  • Research protocol for quasi-experimental crossover trial.

  • Evaluation of 7-day adventure camp for 14–16 year-old secondary school students.

  • Primary outcomes focusing on social and emotional development.

  • Study taking place over three years.

Abstract

This paper presents a research protocol for a quasi-experimental crossover trial of an outdoor adventure program for Year 9 school students in Australia. Previous studies have reported a range of positive outcomes of outdoor camps and adventure programs but warrant cautious interpretation due to limitations in research methods typically employed. This study takes place over a period of 36 months and examines a purpose-designed, seven-day outdoor program (camp) intended to promote positive adjustment in young people. Up to 400 participants (ages 14–16 years) will be recruited from across two Victorian secondary schools. Outcome measures include self-reported social and emotional health, and teacher-reported ratings of academic performance and school conduct. Results will be of interest to educators internationally and those involved in improving social and emotional health in adolescents.

Introduction

Each year millions of adolescents internationally participate in outdoor adventure programs such as camps and journey-based nature experiences, often in the context of regular schooling. The outdoor programs sector, which includes education, government, not-for-profit, and commercial providers, is a large, multi-national industry providing tailored outdoor experiences across all corners of the globe. Motivating this significant level of activity and investment is a belief in the fundamental value of providing young people with the opportunity to engage in, and learn from, undertaking personally challenging activities in natural or wilderness environments.

Consistent with this belief, there is an abundance of anecdotal support for positive effects of participation across all developmental domains, in particular, around social and emotional wellbeing. In an Australian audit of outdoor youth programs conducted by the investigators and industry partners (Williams & Allen, 2012), practitioners who were surveyed (n = 388) identified personal development (e.g., improving self-esteem, confidence, independence, well-being) and social development (e.g., improving relationships, group cooperation) as the two domains where participants in outdoor programs receive greatest perceived benefits. These benefits were cited more often than were benefits related to learning about the environment, or learning about regular classroom subjects (such as mathematics, English, geography, or science). It is noteworthy, however, that only 5% of program leaders cited quality research evidence as a basis for establishing participant benefits, with the vast majority referencing either their own personal observations or feedback from individual participants as the primary support for their practice (Williams & Allen, 2012).

Broadly speaking, the outdoors and camping sector operates on a strong belief that adventure programs have distinctive benefits for personal and social development of adolescents. However, given the extensive use of structured outdoor programs, and the associated costs involved, it is important that such beliefs continue to be tested empirically. Although knowledge gained by individual outdoor program leaders as part of their practice is undoubtedly an essential part of the development of effective practice models, formal studies incorporating more robust research methods are required in order to strengthen objective evidence.

Adolescence is a watershed period in development that profoundly influences transition into adult life and subsequent patterns of functioning, health and wellbeing. As such, the adolescent years provide a unique opportunity to both prevent psychological disorders and positively influence developmental trajectories, with outcomes that will be felt long into the future (Patton et al., 2016).

Improving opportunities for personal and social development in students is of particular interest to schools. The last 20 years has seen a burgeoning interest in the multidisciplinary field of social and emotional learning (Weissberg, Durlak, Domitrovich, & Gullotta, 2015). Teachers and schools have come to recognize that social and emotional learning (SEL) is critical to success in school, work, and life for all students (Bridgeland, Bruce, & Hariharan, 2013). SEL programs designed to promote positive adjustment, as well as prevent problem behaviours, are increasingly appearing in educational settings (Collaborative for Academic, Social, and Emotional Learning, 2015), and a growing body of evidence points to a range of positive outcomes, not only in social and emotional skills, but also in academic performance (Durlak, Weissberg, Dymnicki, Taylor, & Schellinger, 2011).

With a focus on enhancing self-awareness, self-management, social awareness, relationship skills, and responsible decision making (Collaborative for Academic, Social, and Emotional Learning, 2003), school-based SEL interventions target the same areas in which practitioners of outdoor programs report observing benefits. Programs and initiatives for young people that provide opportunities to develop and enhance emotional self-regulation, and that can be delivered on a wide scale, hold considerable appeal and may plausibly lead to savings in delivery of healthcare, welfare, and forensic services during adolescence and in later life. Current and extensive use of structured outdoor programs and camp experiences provides a well-resourced platform for promoting positive development in young people, with a capability for broad-scale implementation through secondary school systems. Results from a plethora of existing non-experimental evaluations and field trials are suggestive of a range of positive effects, including increased self-efficacy (e.g., Sibthorp, 2003) improved self-esteem (e.g., Herbert, 1998), improved rates of abstinence from substance use (Bennett, Cardone, & Jarczyk, 1998), reduced rates of recidivism (Wilson & Lipsey, 2000) and greater internal locus of control (Hans, 2000).

Such findings provide prima facie evidence that outdoor activities have the potential for significant positive impacts on the health and wellbeing of young people. However, interpretations of study findings (including inferences of causality) are frequently hampered by study design limitations including the lack of control groups, an absence of post-program follow-up assessments, failure to report participant attrition, small sample sizes, use of unsophisticated statistical procedures, use of non-validated measurement instruments, and lack of detail regarding participant characteristics and program procedures. Furthermore, very limited research has been conducted examining predictors of program effectiveness, such as attributes of program participants, characteristics of leaders and features of program design. Rigorous development of the research base in this area is therefore critical to transforming the field to one informed by evidence-based practice.

Previous authors have lamented the absence of theory and explanatory models referenced in published articles (e.g. Bandoroff, 1989, Davis-Berman and Berman, 1989, Fletcher and Hinkle, 2002; Friese, Pittman & Hendee, 1995), thereby rendering it difficult to know exactly what processes and mechanisms are being evaluated in many of these studies. More simply, little attention has been given to describe in any detail what happens in outdoor programs and why any included components might lead to the benefits being sought. One model offering scope to inform theory-driven program design and evaluation is the ChANGeS Framework (Williams, 2009).

Based on a review of international adventure programming literature, the ChANGeS Framework identifies five key components of outdoor programs that are thought to be critical for enhancing participant outcomes: Challenge, Activity, Nature, Guided experience, Social milieu (see Fig. 1). Each of these five components represents a range of related factors that might be considered in planning and delivering effective outdoor programs. Examples include: experiences in which participants are extended and have their abilities and personal resources stretched in tasks that may appear at first to lie beyond their reach (Challenge); being actively engaged in a learning environment requiring physical, emotional, cognitive and psychological involvement (Activity); immersion in natural environments characterised by green space, fresh air, freedom from distraction and simple living (Nature); making meaning from experiences through leader guidance and reflection, goal setting, metaphor and debriefing (Guided experience); and being part of a functional community involving small-group living, social modelling, giving and receiving feedback, and negotiating new relationships (Social milieu).

While there is currently no single accepted method of conducting outdoor youth programs, there is some evidence suggesting that various informal combinations of the five ChANGeS components are commonly seen in typical outdoor programs, at least in Australia (Williams & Allen, 2012). In the intervention we describe below, we make explicit use of the ChANGeS Framework in the design and delivery of a new outdoor program aimed at enhancing health and wellbeing amongst adolescent participants.

Given the significant global investment in the area, and the potential for gains in social and emotional learning, new studies employing more robust research methods are needed. This study protocol describes a quasi-experimental, controlled crossover trial of a structured outdoor youth program with secondary school students in Australia. The trial will take place over a period of 36 months and will involve up to 400 young people. The study employs a number of key design features aimed at bolstering confidence in research findings. Results will help inform policy and practice, and lead to more strategic investment in outdoor programs for learning, health promotion and positive youth development.

Section snippets

Study aims & research questions

We set out to evaluate the health and wellbeing impacts of participation in a structured outdoor program amongst Year 9 (ages 14–16 years) secondary school students. The primary aim was to evaluate the effectiveness of a purpose-designed outdoor program in promoting positive development and reducing vulnerability in both the short term (i.e., immediately post-program) and longer term (i.e., six months post-program) using a quasi-experimental research design. A secondary aim was to examine

Study design

This study utilises a quasi-experimental crossover design to assess the extent to which a purpose-built outdoor adventure program might lead to improvements in participants’ resilience, learning and wellbeing. The design involves two secondary schools (A and B), assigned either to an intervention or control arm. In year 1 of the study, students from school A (cohorts A1 and A2) were recruited to an intervention group (participation in an outdoor youth program/camp) while students from school B

Sample size calculations

Power calculations for the study assumed a proposed sample of 480 participants in total, with 240 participants in each study condition. Calculations using the software package GPower3 indicated appropriate power ( > 0.80%) to identify treatment-by-time differences for effect sizes as low as d = 0.13 at the 5% significance level. The same interaction effect for d = 0.17 will still be detectable with power > 80% if 20% of participants drop out in each group.

School recruitment

Two government secondary schools from the Australian state of Victoria were recruited to the study. Recruitment was restricted to government-funded secondary schools that: (1) were not currently offering outdoor programs to students as part of the regular school curriculum (to avoid contamination effects), and (2) could be matched to another secondary school in terms of gender composition and socioeconomic status (to ensure comparability of intervention and control groups). Schools ineligible

Students

Students recruited to intervention groups participated in a seven-day, outdoor camping program during regular school term. During the program students were involved in a range of activities, including challenge tasks, group discussions, remote living and outdoor activities. The outdoor activities included bushwalking, mountain bike riding, canoeing, ropes courses and overnight camping. Students were not expected to have any special experience to take part in the program, and for most activities

Intervention

The primary focus of the study is to examine outcomes associated with participation in an outdoor adventure program (the intervention). We designed a new outdoor program for the purpose of this study, drawing on knowledge from previous research and the collective expertise of the study team and outdoor industry partners.

Three broad principles informed planning decisions regarding the form, design and nature of the outdoor intervention. First, the outdoor program was theoretically informed. The

Measures

Separate surveys were developed for three participant groups involved in the study: i) student participants (both in intervention and control groups) completed an online survey on five occasions; ii) school teachers involved in taking regular classroom sessions with participating students completed a paper-and-pencil survey on two occasions; iii) group leaders delivering the outdoor camps completed a paper-and-pencil survey at the conclusion of each camp. Details of the three surveys are

Pre-Program orientation

Prior to the outdoor programs being delivered, information sessions and camp briefings were presented separately to four different groups: parents, teachers, students, and group leaders. These sessions were designed to facilitate a collaborative approach to program delivery and ensure that all groups were sufficiently prepared to take on their respective roles, and adequately oriented to key principles of our program model.

Analysis Plan

Treatment group differences and differential group changes from baseline to follow-up for outcome measures will be examined using linear mixed modelling. This enables incomplete responses to be analysed under missing at random assumptions (which is reasonable in the current design; Snijders & Bosker, 2012). Investigation of possible effects for order of intervention versus control (i.e., does the student come from a school that participated in the intervention first or the control condition

Ethics

Ethics approval for this project was received from the University of Melbourne Behavioural and Social Science Human Ethics Sub-Committee (ID: 144318), and the Victorian Department of Education and Training (ID: 2014_002549).

Workplan & Timeline

A breakdown of tasks and proposed timelines across the three-year study is shown below in Table 2.

Project Governance and Personnel

The Outdoor Youth Programs Research Alliance (OYPRA) is an Australian group established to investigate and document the benefits of outdoor programs for young people. The Alliance was founded in 2009 with the aim of developing a program of research that evaluates the potential benefits of outdoor, camping and nature-based programs on the resilience, learning and wellbeing of young people. The team includes representatives from the outdoor education and recreation industry, the health research

Funding

This work is supported by an Australian Research Council (ARC) Linkage Grant (LP130100314), which offers matched funding to industry partner contributions. This scheme provides funding through a competitive, peer-reviewed application process.

Acknowledgements

We would like to thank staff, parents, and students of participating schools for their support of this research project. We would also like to thank members of OYPRA, and the wider Australian outdoor sector for their ongoing support.

References (47)

  • J. Bridgeland et al.

    The missing piece: A national teacher survey on how social and emotional learning can empower children and transform schools A Report for CASEL

    (2013)
  • Collaborative for Academic, Social, and Emotional Learning

    Safe and sound: An educational leader’s guide to evidence-based social and emotional learning(SEL)programs

    (2003)
  • Collaborative for Academic, Social, and Emotional Learning

    CASEL guide: Effective social and emotional learning programs – Middle and high school edition

    (2015)
  • J.M. Cramm et al.

    The importance of general self-Efficacy for the quality of life of adolescents with chronic conditions

    Social Indicators Research

    (2013)
  • J. Davis-Berman et al.

    The Wilderness Therapy Program: An empirical study of its effects with adolescents in an outpatient setting

    Journal of Contemporary Psychotherapy

    (1989)
  • J.A. Durlak et al.

    The impact of enhancing students’ social and emotional learning: A meta-analysis of school-based interventions

    Child Development

    (2011)
  • T.B. Fletcher et al.

    Adventure based counseling: An innovation in counseling

    Journal of Counseling & Development

    (2002)
  • G.T. Friese et al.

    Studies of the use of wilderness for personal growth, therapy, education, and leadership development

    An annotation and evaluation

    (1995)
  • M. Gagne

    The role of autonomy support and autonomy orientation in prosocial behavior engagement

    Motivation and Emotion

    (2003)
  • R. Goodman

    The strengths and difficulties questionnaire: A Research note

    The Journal of Child Psychology and Psychiatry

    (1997)
  • K.T. Han

    Responses to six major terrestrial biomes in terms of scenic beauty, preference, and restorativeness

    Environment & Behavior

    (2007)
  • T.A. Hans

    A meta-analysis of the effects of adventure programming on locus of control

    Journal of Contemporary Psychotherapy

    (2000)
  • D.J. Hawes et al.

    Australian data and psychometric properties of the Strengths and Difficulties Questionnaire

    Australian and New Zealand Journal of Psychiatry

    (2004)
  • Cited by (7)

    View all citing articles on Scopus
    View full text