Elsevier

Psychiatry Research

Volume 206, Issues 2–3, 30 April 2013, Pages 293-301
Psychiatry Research

Development of a measure quantifying adverse psychotherapeutic ingredients: The Experiences of Therapy Questionnaire (ETQ)

https://doi.org/10.1016/j.psychres.2012.11.026Get rights and content

Abstract

While psychotherapies are of established value, they may, as active treatments, risk adverse outcomes. As there is no validated measure of potentially negative psychotherapeutic ingredients, we sought to develop such a measure for use in psychotherapy evaluation studies. Based on a review of the literature, a 103-item experiential measure was derived. Psychometric properties and scale score correlates were examined in a sample of more than 700 respondents. Principal component analyses revealed a five-factor solution, explaining 53.4% of the variance; namely ‘Negative Therapist’, ‘Pre-occupying Therapy’, ‘Beneficial Therapy’, ‘Idealization of Therapist’ and ‘Passive Therapist’ constructs. Derived factors had high internal consistency, and scale scores were correlated with a number of clinically relevant demographic and treatment characteristics. An independent study established high test–retest reliability for the measure. Assessment of any adverse effects of psychotherapy is of clinical and research significance. We report the development of a measure that should allow the impact of such effects to be quantified in treatment studies, and especially in apportioning the contribution of such non-specific therapeutic effects.

Introduction

While psychotherapies are widely practiced, have a supportive evidence base and are valued by many patients, there is increasing recognition that, as for all active treatments, they can risk adverse outcomes. While many studies (Rogers, 1957, Frank, 1972, Orlinsky and Howard, 1975, Burns and Nolen-Hoeksema, 1992, Keijsers et al., 2000, Kim et al., 2006) have identified or suggested positive (or ‘non-specific’) ingredients of beneficial psychotherapy (e.g. empathy, non-possessive warmth, provision of an explanatory model, inspiring hope and redressing demoralisation), there has been little systematic consideration of any actual or potential adverse components, while currently there are no tools specifically quantifying adverse aspects of psychotherapy. “Adverse events” do not necessarily equate to treatment ‘failure’ (which may occur for reasons unrelated to the therapist per se)—rather, such events can represent situations where the individual felt they had to terminate therapy based on the belief that the therapy itself or perhaps the therapist were having an adverse impact on them. We previously theoretically reviewed (Berk and Parker, 2009) the domain and the few salient studies, and argued the advantages of being able to measure and quantify adverse events occurring during psychotherapy. We now briefly overview those issues and then describe the development of such a measure.

The possibility of psychotherapy having adverse ‘side-effects’ has rarely been considered, with Nutt and Sharpe (2008) noting that there is an “assumption…that as psychotherapy is only talking….no possible harm could ensue.” Any adverse impact could theoretically reflect ‘therapy’ or ‘therapist’ factors. Therapy factors could include the inappropriate use of psychotherapy (e.g. prioritising psychotherapy when medication is the treatment of choice) or the use of interventions identified as potentially harmful (e.g. Critical Incident Stress Debriefing for Post-traumatic Stress Disorder; Lilienfield, 2007). Such a domain is predictably extremely difficult to quantify-with our focus instead being on therapist factors. In that regard, Foulkes (2010) highlighted the importance of differentiating treatment-based clinician effects and those reflecting the style of the individual clinician.

In terms of such an individual therapist ‘impact’ issue, the psychotherapist brings technical strategies into psychotherapy, but, in addition, effectively ‘prescribe’ themselves by providing (or not providing) the non-specific therapeutic factors noted earlier that have been identified as being central to successful psychotherapy (Lambert, 1992, Wampold, 2001). The latter quantified that only 8% of the variance in outcome reflected specific therapeutic ingredients, and that non-specific ‘common factors’ accounted for nine times more outcome variability than specific ingredients. In their review of the therapeutic alliance, Elvins and Green (2008) noted that the therapist – even when supervised and monitored in taking up particular approaches – significantly contributes to variability in treatment outcome as a consequence of the degree to which they express such components. Furthermore, the quality of the relationship that develops between therapist and client (the ‘therapeutic alliance’) has been shown to reliably predict clinical outcome (Ardito and Rabellino, 2011). Finally, adverse outcomes may also be attributed to extra-therapeutic factors, such as client factors and health system factors (Lambert and Ogles, 2004).

While specific and non-specific ingredients have long been a focus of success or improvement in psychotherapeutic research, our focus is on any converse propensity. Psychotherapists have long conceded a number of potentially contributory factors to poor therapy and to clinical ‘failure’, including the therapist failing to listen to the patient and more following their own agenda; making the same mistake multiple times; inflexibility and reluctance to make necessary adjustments; not having a clear direction; over-confidence and narcissism; an internal feeling of ineptitude; failure to create a solid alliance; losing control of self or counter-transference issues; and making invalid assumptions (Kottler and Carlson, 2003). We therefore judged that it would be useful to develop a measure of adverse psychotherapeutic parameters that, in turn, might risk adverse outcomes. We pursued constructs that might be specific to psychotherapy ‘type’ (e.g. long-term psychotherapy might engender dependency, cognitive approaches might lead to more ruminative worry) but more focussed on ones that reflected therapist ‘style’ contributions. In part, we hypothesised that if identified non-specific factors provide a powerful positive benefit to therapy, their absence (whether by acts of omission or commission) should influence therapy in an adverse way. We now describe the development of the measure, overview some of its properties, and consider potential applications.

Section snippets

Methods

We reviewed studies referenced earlier of psychotherapeutic ingredients that, when present, enhance benefits of psychotherapy but, when absent, risk adverse outcomes (e.g. factors affecting the forming and maintenance of an alliance, the extent to which the therapy appeared structured, the extent to which the therapist encourages dependency or enmeshment). We reframed such components as absent or limited in writing the item descriptions, and added more overt deleterious therapist behaviors

Results

For those currently receiving therapy, 707 individuals commenced the questionnaire online, and with 360 (50.9%) completing all questions. Similarly, for those who had received therapy in the past, 356 (52.3%) of the 680 who commenced the questionnaire completed the questionnaire in full. Our results are restricted to those who completed all questions, detailed below.

Discussion

Based on our literature review (Berk and Parker, 2009), the Experiences of Therapy Questionnaire (ETQ) was developed to quantify adverse effects associated with psychotherapy. Whilst the dominant approach in most psychotherapy dissection and therapy alliance studies has been to identify the ‘positive’ non-specific ingredients, we opted to explore the opposite—identifying constructs associated with adverse psychotherapy.

ETQ factor structure and linkages were analysed in a large sample of those

Disclosure

Gordon Parker has received financial support or reimbursement from six pharmaceutical companies (Eli Lilly, Servier, Astra Zeneca, Pfizer, Lundbeckand Glaxo SmithKline) over the last three years, for lectures, chairing of meetings, membership of advisory groups or attendance at conferences, and received research grant funding from the Commonwealth Department of Health and Ageing, the Rotary and a number of Black Dog Institute donors.

Kathryn Fletcher is funded under Program Grant 510135 from the

Acknowledgements

The authors thank Dusan Hadzi-Pavlovic for statistical advice and Lesley Berk for her assistance with the manuscript.

References (37)

  • F. Borge et al.

    Predictors of outcome in residential cognitive and interpersonal treatment for social phobia: do cognitive and social dysfunction moderate treatment outcome?

    Journal of Behavior Therapy and Experimental Psychiatry

    (2010)
  • R. Elvins et al.

    The conceptualization and measurement of therapeutic alliance: an empirical review

    Clinical Psychology Review

    (2008)
  • R.B. Ardito et al.

    Therapeutic alliance and outcome of psychotherapy: historical excursus, measurements, and prospects for research

    Frontiers in Psychology

    (2011)
  • M.S. Bartlett

    A note on the multiplying factors for various chi square approximations

    Journal of the Royal Statistical Society

    (1954)
  • M. Berk et al.

    The elephant on the couch: side-effects of psychotherapy

    Australian and New Zealand Journal of Psychiatry

    (2009)
  • C. Brykczyńska

    Changes in the patient's perception of his therapist in the process of group and individual psychotherapy

    Psychotherapy and Psychosomatics

    (1990)
  • D.D. Burns et al.

    Therapeutic empathy and recovery from depression in cognitive-behavioral therapy: a structural equation model

    Journal of Consulting and Clinical Psychology

    (1992)
  • D. Castle et al.

    Group-based psychosocial intervention for bipolar disorder: randomised controlled trial

    British Journal of Psychiatry

    (2010)
  • D.J. Castle et al.

    Psychosocial interventions for bipolar disorder

    Acta Neuropsychiatrica

    (2009)
  • F. Colom et al.

    Has number of previous episodes any effect on response to group psychoeducation in bipolar patients? A 5-year follow-up post hoc analysis

    Acta Neuropsychiatrica

    (2010)
  • P. Dignam et al.

    Detached from attachment: neurobiology and phenomenology have a human face

    Acta Neuropsychiatrica

    (2010)
  • P. Foulkes

    The therapist as a vital factor in side-effects of psychotherapy

    Australian and New Zealand Journal of Psychiatry

    (2010)
  • J. Frank

    Common features of psychotherapy

    Australian and New Zealand Journal of Psychiatry

    (1972)
  • J.L. Horn

    A rationale and test for the number of factors in factor analysis

    Psychometrika

    (1965)
  • E.A.G. Joosten et al.

    Systematic review of the effects of shared decision-making on patient satisfaction, treatment adherence and health status

    Psychotherapy and Psychosomatics

    (2008)
  • H. Kaiser

    A second generation Little Jiffy

    Psychometrika

    (1970)
  • H. Kaiser

    An index of factorial simplicity

    Psychometrika

    (1974)
  • G. Keijsers et al.

    The impact of interpersonal patient and therapist behavior on outcome in cognitive-behavior therapy. A review of empirical studies.

    Behavior Modification

    (2000)
  • Cited by (32)

    • Negative effects in internet-based interventions for depression: A qualitative content analysis

      2021, Internet Interventions
      Citation Excerpt :

      Although the amount of therapeutic guidance per se was not a significant predictor for the experience of negative effects, the quality of the therapeutic relationship was associated with the experience of negative effects, as participants with lower ratings of the working alliance were more likely to report the occurrence of negative effects. This is in line with prior research that emphasized the association of a negative therapeutic relationship and negative effects (Parker et al., 2013). Moreover, in a content analysis of client e-mails in guided IBI for depression Svartvatten et al. (2015) found significant positive correlations between participants' statements highlighting a warm and trusting relationship and a change in outcome.

    • Deterioration rates in Virtual Reality Therapy: An individual patient data level meta-analysis

      2019, Journal of Anxiety Disorders
      Citation Excerpt :

      Furthermore, several self-report measures have been developed over the years (e.g. Hatfield, McCullough, Frantz, & Krieger, 2010; Ladwig, Rief, & Nestoriuc, 2014). Among the developed instruments are, for instance, the Vanderbilt Negative Indicator Scale (VNIS; Suh, Strupp, & O’Malley, 1986), the Inventory for the Assessment of Negative Effects in Psychotherapy (INEP; Ladwig et al., 2014) the Unwanted Events and Adverse Treatment Reaction Checklist for Psychotherapy (UE-ATR; Linden, 2013), the Experience of Therapy Questionnaire (ETQ; Parker et al., 2013) and the Negative Effects Questionnaire (NEQ; Rozental, Kottorp, Boettcher, Andersson, & Carlbring, 2016). However, as explained in detail by Rozental et al. (2016), each scale has its own theoretical background as well as strengths and weaknesses.

    • Side Effects in Psychodynamic and Cognitive Behavior Therapy

      2024, Journal of Contemporary Psychotherapy
    View all citing articles on Scopus
    View full text