Nine Forward–Backward Translations of the Hopkins Symptom Checklist-25 With Cultural Checks

Introduction: The Hopkins Symptom Checklist-25 (HSCL-25) is an effective, reliable, and ergonomic tool that can be used for depression diagnosis and monitoring in daily practice. To allow its broad use by family practice physicians (FPs), it was translated from English into nine European languages (Greek, Polish, Bulgarian, Croatian, Catalan, Galician, Spanish, Italian, and French) and the translation homogeneity was confirmed. This study describes this process.

Methods: First, two translators (an academic translator and an FP researcher) were recruited for the forward translation (FT). A panel of English-speaking FPs that included at least 15 experts (researchers, teachers, and practitioners) was organized in each country to finalize the FT using a Delphi procedure.

Results: One or two Delphi procedure rounds were sufficient for each translation. Then, a different translator, who did not know the original version of the HSCL-25, performed a backward translation in English. An expert panel of linguists compared the two English versions. Differences were listed and a multicultural consensus group determined whether they were due to linguistic problems or to cultural differences. All versions underwent cultural check.

Conclusion: All nine translations were finalized without altering the original meaning.

Introduction

How to manage people with depression in primary care is a growing challenge worldwide. Indeed, Family practice physicians (FPs) are at the frontline, while secondary care services are increasingly under threat (1–4). Depression manifests (for laypersons) itself in various ways: (i) as a syndromic “disorder” in which contextual distress, anxiety, and somatoform disorders overlap; (ii) as a suffering that is difficult to express, acknowledge, and discuss; and (iii) as a long-term condition with subjective and objective features that can be measured (5). Due to these inter-individual variabilities, FPs may experience difficulties in detecting depression and may easily misjudge the symptoms and their intensities, if they do not use formal instruments (6, 7). Moreover, the depression incidence and prevalence rates differ widely in family practice, due to complex contextual variations, differences in healthcare systems, concepts of disorder, objectives, and practices, as well as cultural variations in symptom expression (8, 9). These difficulties may lead to inappropriate care and potential side effects due to drugs' use as well as public health issues (10–12). A short discussion of the results obtained using a relevant questionnaire is often the first step toward an open dialogue with the patient.

Collaborative primary care mental health models can improve the management of patients with depression. To this aim, the European General Practice Research Network (EGPRN) developed a collaborative research agenda (13). Specifically, the EGPRN adopted a standardized methodology in which European FPs experts from different healthcare systems and who speak different languages and have different cultural references set up an established consensus procedure to identify reliable, standardized, efficient, and ergonomic tools for depression assessment that take into account cultural and linguistic differences (14–17). These tools need to be accepted by both FPs and psychiatrists to improve collaboration (18). They must be feasible in the FP's surgery, in primary or psychiatric care, and also suitable for research purposes (19). Finally, they must be validated and reliable.

A handbook was developed to guide the selection of a single tool that would be then translated into different languages, using a forward and backward translation procedure (inspired by Brislin's translation model). This is a consensual procedure that has been used in other cross-cultural studies (20–22). At each step, the key points and purposes were debated and decided by consensus among the involved European experts. First, a systematic literature review, according to the PRISMA criteria, allowed the identification of seven tools that had been validated against a psychiatric examination using the DSM-IV or DSM-5 major depression criteria (23). Then, a consensus procedure (RAND/UCLA Appropriateness Method) led to the selection of one tool on the basis of its effectiveness, reliability, and ergonomics (24): the self-report Hopkins Symptom Checklist-25 (HSCL-25) (23–26). This is a validated, reliable diagnostic tool to assess (27, 28) the presence and severity of anxiety and depression symptoms during the previous week (29, 30). Its specificity compared with clinical interview is robust: between 0.78 to 0.88, the reliability (Alpha de Cronbach) is between 0.87 to 0.97 (31). The HSCL-25 short length self-administered format is perfectly suited for use in busy primary care settings with many competing demands. It may represent a practical instrument to alert FPs to potentially depressive or anxious symptomatology.

A qualitative procedure with the FP's involvement was necessary to obtained that were linguistically and culturally equivalent to the original version, ecologically embedded in primary care.

The objective of the present study was to translate the HSCL-25 into the languages of the different team members, without losing homogeneity, and in a language suitable to the primary care context (22, 32).

Materials and Methods

This three-step standardized study included: (i) forward translation (FT), (ii) backward translation (BT), and (iii) cultural check (8, 33, 34) (Figure 1).

FIGURE 1

Figure 1. The translation procedure. CE, cultural effect; BTP, backward translation problems; FTP, forward translation problems.

The FT was carried out with an incorporated Delphi consensus procedure (35–37). This is a systematic, interactive method that involves a panel of experts using iterative procedures (38) and that allows reaching consensus in a rigorous way (39–41). This process requires:

• Anonymity of participants to ensure response reliability and avoid contamination,

• Iteration, which allows participants to refine their views in the light of the group work progress,

• Feedback control under the investigator's responsibility,

• Statistical aggregation of the group's responses to allow a quantitative and qualitative analysis of data (42–45).

The EGPRN French team ensured that this protocol was followed throughout the process. The FT of the different HSCL-25 items had to be validated daily by the expert panel, composed of EGPRN members, all actively involved in the process.

Briefly, for each language, the National Investigators (NI) selected translators knowledgeable about healthcare terminology to organize two translation (FT and BT) teams who were blind to the other team's work. The FT team included one member of the FP research group and one official translator for each country. The BT team involved one (or two) FPs and one official translator (22).

The NIs also recruited a panel of FP experts in their own countries, anonymized the experts' responses, and allocated an identification number for later identification (42). Initially, 20 to 30 experts were recruited per country to secure the presence of at least 15 participants till the project end. The FP experts were selected using the following inclusion criteria: native of their country of residence and speaking their native language, and fluent in English (32). At least half of them had to be involved in teaching and/or research activities. To assess the panel representativeness of their country FPs, the experts provided the following information: sex, practice type, years of practice, and publication record (46).

According to the Brislin's Guidelines for the Process of Cross-Cultural Adaptation of Self-Report Measures, once the FT was completed, a BT was performed with two goals: (i) to ensure the identification of language issues and (ii) to detect translation problems linked to cultural adaptation issues. Indeed, as translation biases related to cultural aspects of each country were possible, a cultural check was required to ensure homogeneity (17, 20, 33, 34, 47). To this aim, in each country, an FP researcher and a linguist analyzed all BT propositions and compared them with the original HSCL-25 version to establish whether there was any significant difference in terms of meaning. Their report was submitted to a consensus group whose task was to clarify the nature of each FT-BT discrepancy from three problem areas: (i) BT problems were eliminated if the difference was explained by an incorrect BT; (ii) FT problems were defined as an anomaly in transcribing the original English (semantic/idiomatic differences relative to the original English version); and (iii) cultural effects (CE) were considered validated if there was no linguistic problem with the translation, but the item needed to be modified to be understood by the patients in their own “everyday” language (Figure 1).

This led to a linguistically stable, definitive translation that maintained the HSCL-25 meaning (i.e., structure and question order and method of use) for each involved country.

Ethical request: The EGPRN French team was in charge of checking the volunteering process and confirming the absence of potential conflicts of interest for all participants. The Ethics Committee of the approved the whole process.

The EGPRN French team recruited all NIs and obtained their consent, managed the voluntary participation in the study and produced an absence of conflict-of-interest statement.

Each NI asked participants to sign the informed consent.

Results

NI Panel Description

The NI panel included 11 NIs (including n = 8 women) from eight European countries. They were all FPs, EGPRN members, and fluent in English. Ten NIs practiced in urban areas of more than 5,000 inhabitants and one worked in an urban area with 2,000–5,000 inhabitants. Eight had also teaching duties in addition to being researchers (total number of publications by the panel members: 152). The mean number of years of practice and of research were 21.3 and 12.4 years, respectively. In the panel, two NIs were from two distinct cultural regions of coastal Spain (Catalonia and Galicia), and two were Croats. The other countries were each represented by a single NI (Table 1).

TABLE 1

Table 1. National investigators' panel.

Forward Translation

For the Delphi consensus procedure, 14 (Germany) to 31 experts (Spain) were recruited. In compliance with the selection criteria, they were all FPs and fluent in English. The expert panel included 215 FPs (111 men and 104 women). Among them, 20 worked in a city of <2,000 inhabitants, 36 in a city with 2,000–5,000 inhabitants, and 159 in a city with >5,000 inhabitants. Their clinical experience was analyzed according to years of practice (mean: 16.4 years of experience) (Table 2).

TABLE 2

Table 2. Characteristics of each country expert panel.

In Poland, Bulgaria, Germany, Spain, and the Catalonia region of Spain, there was only one Delphi round, and two rounds in the other countries. Almost all translation proposals for each item of the HSCL-25 questionnaire were accepted in one round (273/320: 85.3%) (Table 3). The other proposals for which consensus was not reached went through a second round. The NI and the forward official translator synthesized the experts' comments to produce a new translation proposition for the second round.

TABLE 3

Table 3. Results of the first Delphi round.

Some Translation Issues Required a Second Proposal and Another Delphi Round

In Croatian, eleven proposals were rejected in the first round. For example, for item #17 (“Feeling blue”), the first proposal was “Bili ste tužni,” which was considered to be too focused on melancholia, and was modified to “Bili ste sjetni,” closer to the concept of sadness. All new proposals were accepted during the second round.

As a German version of the HCL-25 was already available, the German NIs proposed that their expert panel would discuss this version, without producing a new FT. All items were accepted in the first Delphi round. At this step, the German NIs stopped the procedure. No cultural check was performed.

Nine Greek proposals were rejected in the first round. For example, for item #1 (“Being scared for no reason”): the first proposal “Είμαι τρομοκρατημένος χωρίς αιτία” was considered too strong. Consensus was reached on the second proposal: “Είμαι τρομαγμένος χωρίς αιτία.” All new proposals were accepted during the second round.

In the French translation, consensus was not reached on 18 proposals in the first round and needed further specification in the second round. For example, for item #25 (“Sleep disturbance”), the first proposal was “Vous n'arrivez pas à dormir” that was modified to “Votre sommeil était perturbé,” closer to the English word: “disturbance.” All new proposals were accepted during the second round.

In the Italian translation, consensus was not reached on five proposals during the first round. For example, for item #5 (“Heart racing”), the first proposal “Avere tachicardia” was considered too focused on clinical symptoms and was modified to “Sentire il cuore battere veloce,” which was more familiar according to the reviewers. All new proposals were accepted during the second round.

In the Spanish Galician translation, consensus was not reached on three proposals in the first round. For example, for item #6 (“Trembling”), the first proposal was “Trema,” the present indicative of the verb “Tremar.” The second proposal was “Ten tremores” and was accepted in the second round. All new proposals were accepted during the second round.

Backward Translation and Cultural Check

The initial instructions, the 25 items, the quotation and the explanatory sentences were all back-translated into English by the BT team. In total, 36 propositions were analyzed. All BTs were compared linguistically to the original. Differences were noted for submission to the NIs and the consensus group. Three consensus group meetings were necessary with national feedback between each. The main adaptations, produced as a result of national feedback and the consensus resulting from the cultural check, are described below.

By Languages and Language Groups

Croatia: 8 items were different (2 were BT problems, and 8 required a cultural adaptation).

The main cultural aspect was the use of the present perfect, which is a tense of state and not of action, commonly employed in daily life. Therefore, in items #2, 7, 9, and 10, “feeling” was replaced by “you have been.” Only one item seemed to be stronger than in the original version. Indeed, “Faintness,” was replaced by “Weakness,” but in Croatian this is equivalent to faintness.

Bulgaria: 3 items were different (2 were BT problems, and 1 required a cultural adaptation).

“Feeling low in energy” became “A sense of low energy.” Overall, the Bulgarian translation was the most stable among the three Slavic languages.

Poland: 13 items were different (7 were BT problems, and 6 required a cultural adaptation).

Most problems resulted from a conceptual issue. For instance, in Polish, “Heart racing” became “Palpitations,” “Trembling” became “Tremors,” and “An effort” was translated into “A burden.” “Headache” was translated into “Headaches” in Polish for grammatical reasons.

In all three slavic languages (Croatian, Bulgarian, and Polish), “Feeling restless” was translated into “Anxiety” because there is no equivalent word to express these ideas. A word-by-word translation, in that case, was impossible.

For the Greek language, the translation was mainly based on an adaptation according to gender. The experts concluded that there was a general CE affecting all parts of the scale. However, no real difference in meaning was detected, and the Greek HSCL-25 scale remained stable relative to the original.

France: 5 items were different (4 were BT problems, and 1 required a cultural adaptation).

For the French scale, the present tense is normally used in everyday language. However, the past tense was used in the FT. In everyday life French, the past tense is considered an older, upper-class language style. Therefore, all tenses were modified. For instance, “Tout était un effort pour vous” became “Tout est un effort pour vous” in the final version.

Italy: 7 items were different (6 were BT problems, and 1 required a cultural adaptation).

In the Italian scale, the male plural form was used because this is the usual way of speaking/writing; the translation had to be modified according to gender.

Spain: 6 items were different (1 was a BT problem, and 5 required a cultural adaptation).

“Feeling no interest” was translated in “No siente interes por nada” in standard Spanish, and “Worthless feeling” became “Feeling useless.” However, in Standard Spanish, “inutil” means also “worthless.”

Catalonia: 7 items were different (4 were BT problems, and 3 required a cultural adaptation).

Galicia: 5 items were different (1 was a BT problem, and 4 required a cultural adaptation).

In the Galician scale, item #14 “Losing sexual interest,” was translated into “Loss of sexual interest” that expresses a state, and not an action (the original English version); however, the local experts considered it a normal way of speaking/writing in that language.

In the Galician and Catalan translations, “Blame oneself” turned into “Blame yourself” in the BT because the term “oneself” is not commonly employed.

For the Hispanic languages, the translation had to be modified according to gender. The item “Faintness” was translated into “Weakness” (e.g., “Debilidad,” “Debilitat,” and “Debilidade” in standard Spanish, Catalan and Galician respectively). Similarly, the item “Heart racing” was translated into “Palpitations” (i.e., “Palpitaciones” and “Palpitacions” in the standard Spanish and Galician versions).

For All of Languages

Item #17 “Feeling Blue” generated a CE in six of the nine languages. A word-by-word rendition was impossible and required a cultural adaptation.

Items #15 “Feeling lonely,” #18 “Thinking of ending one's life,” #19 “Feeling trapped” and #25 “Sleep disturbance” remained stable after the BT.

Concerning the scale instructions and the quotation question, the BT was different from the original version in nine items, except the explanation concerning the time required to fill in the scale. Many translation problems were related to “cultural” effects. For example: in French, some terms were replaced by typical expressions commonly employed in questionnaires: e.g., “pencil-and-paper” was translated into “auto questionnaire” and “Not at all” by “Pas du tout d'accord.”

Interestingly, there were translation similarities (often with stronger meanings or medical connotations) not only among languages belonging to the same linguistic group, but also among languages from different groups. The best example concerns item #3 “Faintness” that was translated into “Weakness” in Catalan, Standard Spanish, Galician, and also in Croatian, a term with a more prosaic than medical connotation.

At the end of the cultural analysis, the consensus group finally concluded that the meaning was not changed, and the translation was finalized in all nine languages (see Tables 4–6).

TABLE 4

Table 4. Final translation of the HSCL-25 in nine European languages: items 1–25.

TABLE 5

Table 5. Final translation of the HSCL-25 in nine European languages: scale instructions.

TABLE 6

Table 6. Final translation of the HSCL-25 in nine European languages: general instructions.

Discussion

Using a three-step qualitative procedure, ecologically embedded in primary care, nine consensual translations of the HSCL-25 were obtained that were linguistically and culturally equivalent to the original version, in three language families (Hellenic, Slavic, and Romance). A German version already existed. The aim of this procedure was to meticulously track inconsistencies between local translations that could lead to misinterpretation. This methodical and transcultural validation ensured the transfer of the same content from one language to another and its reliability (17, 47).

The Greek translation remained the most stable, followed by Bulgarian. Item #17, “Feeling blue” was the most challenging to translate, followed by item #3 “Faintness” and item #5 “Heart racing.” Some scales needed adaptations in terms of tense (French, Croatian) and in terms of gender (Greek, Italian, and Hispanic languages).

Research and Teaching Implications

Translation remains the most crucial step in the adoption of an instrument developed in another nation using a different language. Errors in translation may distort the original intent of the instrument, thus compromising its validity and reliability (48). Semantic issues might affect comparability in international studies because the same word is interpreted differently across countries and cultures (49, 50). Moreover, some terms and concepts may not exist in other languages or may have additional connotations that backward translations do not always reveal. Challenges arise not only because of the word-to-word literal translation, but also because of the linguistic form of the language, such as tone and syntax (51).

These nine translations of the HSCL-25 are now linguistically similar, in terms of meaning, compared to the original version. However, they need further testing because this first step is not sufficient to complete the task of translating them and supporting their cross-cultural validity. The external and internal validity of each version has to be evaluated to ensure that their reliability is comparable with that of the original version. This will be achieved through quantitative studies in primary care daily practices (52).

In most European countries, FPs can now use this tool for family practice research studies and for assessing depression severity in their patients. The use of such a shared tool may have a great impact on the feasibility of future research on depression in primary care. It will facilitate data comparison among European countries and consequently it will allow statistical reviews on depression epidemiology and symptoms throughout Europe. The use of the same instrument can support the conceptualization of the studied phenomenon across different studies, and the findings can then be compared (21).

Limitations

A key point of this study was the FPs' involvement in the translation to reduce the selection bias and to ensure the sample quality nevertheless as in all formalized expert consensus procedure a selection bias of the experts remained possible. Our experts' sample was constructed purposively and if we did our best to avoid a selection bias it remained possible. As described by many translators when discussing scientific translation work, a “specialist” in the field (e.g., primary care daily practice in this case) should take a last look at the translation (20, 53, 54) and become the main arbiter of the quality of the final translation (55). Thus, specific attention was paid in choosing FP researchers and certified bilingual translators with sufficient knowledge of healthcare terminology a selection bias was still possible.

The cultural control check was as consistent as possible. It involved a careful step-by-step analysis to prevent confusion bias and linguistic problems. The formalized consensus method allowed the gradual evaluation of each item to strengthen the accuracy of the validated translations and designing the end-result. Nevertheless, an information or a confusion bias remained possible. Our results should be interpretated in the light of these limitations.

Conclusion

A translation of the HSCL-25 in which homogeneity is ensured is now available for Spain and its culturally distinct regions of Galicia and Catalonia, and also for France, Greece, Italy, Poland, Bulgaria, and Croatia. It is now ready to be tested in actual and representative primary care populations to further validate its test-parameters.

Data Availability Statement

The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

Ethics Statement

The studies involving human participants were reviewed and approved by CPP (Protection of Persons Committee) of the University Hospital of Brest. Reference CPP: CPP Ouest VI 872; Study ID RCB: n°2014-A01790-47. The patients/participants provided their written informed consent to participate in this study.

Author Contributions

PN designed the study, collected data, led meetings, drafted the article, and submitted it for publication. JL designed the study, collected data, attended meetings, and reviewed the article. MG-L and BL reviewed the article. RA, DK, SC, MH, HL, AC, MR-B, AS, SA, and CL participated as national investigator. SS-S participated as co-national investigator. TM reviewed the article and gave final approval for the version to be published. HV and PV designed the study, reviewed the article, and gave final approval for the version to be published. All authors contributed to the article and approved the submitted version.

Funding

This research was supported by the National Institute for Health Research (NIHR) Applied Research Collaboration Kent, Surrey, Sussex.

Author Disclaimer

The views expressed are those of the author(s) and not necessarily those of the NHS, the NIHR or the Department of Health and Social Care.

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher's Note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Acknowledgments

We would like to thank all FPs who participated in the research process throughout Europe and all trainees in Family Medicine from Brest University who participated in the research process and our proof-reader Mrs. Alex Gillman for her accurate translations.

Abbreviations

BT, backward translation; CE, cultural effect; DSM, diagnostic and statistical manual of mental disorders; EGPRN, European general practice research network; FPs, family practice physicians; FT, forward translation; PRISMA, preferred reporting items for systematic reviews and meta-analyses; RAND, research and development; RAND/UCLA, research and development/University of California Los Angeles.

References

1. King M, Nazareth I, Levy G, Walker C, Morris R, Weich S, et al. Prevalence of common mental disorders in general practice attendees across Europe. Br J Psychiatry. (2008) 192:362–7. doi: 10.1192/bjp.bp.107.039966

PubMed Abstract | CrossRef Full Text | Google Scholar

2. Ayuso-Mateos JL, Vázques-Barquero JL, Dowrick C, Lehtinen V, Dalgard OS, Casey P, et al. Depressive disorders in Europe: prevalence figures from the ODIN study. Br J Psychiatry. (2001) 179:308–16. doi: 10.1192/bjp.179.4.308

PubMed Abstract | CrossRef Full Text | Google Scholar

3. Alonso J, Angermeyer MC, Bernert S, Bruffaerts R, Brugha TS, Bryson H, et al. Prevalence of mental disorders in Europe: results from the European Study of the Epidemiology of Mental Disorders (ESEMeD) project. Acta Psychiatr Scand Suppl. (2004) 109:21–7. doi: 10.1111/j.1600-0047.2004.00327.x

PubMed Abstract | CrossRef Full Text | Google Scholar

4. Torzsa P, Szeifert L, Dunai K, Kalabay L, Novák M. Diagnosis and therapy of depression in family practice}. Orv Hetil. (2009). 150:1684–93. doi: 10.1556/oh.2009.28675

PubMed Abstract | CrossRef Full Text | Google Scholar

5. Jorm AF. Mental health literacy. Public knowledge and beliefs about mental disorders. Br J Psychiatry. (2000) 177:396–401. doi: 10.1192/bjp.177.5.396

PubMed Abstract | CrossRef Full Text | Google Scholar

6. Mitchell AJ, Vaze A, Rao S. Clinical diagnosis of depression in primary care: a meta-analysis. Lancet. (2009) 374:609–19. doi: 10.1016/S0140-6736(09)60879-5

PubMed Abstract | CrossRef Full Text | Google Scholar

7. Ani C, Bazargan M, Hindman D, Bell D, Farooq MA, Akhanjee L, et al. Depression symptomatology and diagnosis: discordance between patients and physicians in primary care settings. BMC Fam Pract. (2008) 9:1. doi: 10.1186/1471-2296-9-1

PubMed Abstract | CrossRef Full Text | Google Scholar

8. Cuéllar I, Paniagua FA. Handbook of Multicultural Mental Health : Assessment and Treatment of Diverse Populations. San Diego, CA: Academic Press (2000).

Google Scholar

9. Marsella AJ, Yamada AM. Culture and mental health: An introduction and overview of foundations, concepts, and issues. In: Cuéllar I, Paniagua FA, editors. Handbook of Multicultural Mental Health. San Diego, CA: Academic Press (2000). p. 3–24.

Google Scholar

10. Alonso J, Angermeyer MC, Bernert S, Bruffaerts R, Brugha TS, Bryson H, et al. Psychotropic drug utilization in Europe: results from the European Study of the Epidemiology of Mental Disorders (ESEMeD) project. Acta Psychiatr Scand Suppl. (2004) 109:38–46. doi: 10.1111/j.1600-0047.2004.00331.x

PubMed Abstract | CrossRef Full Text | Google Scholar

11. Wells KB. Caring for depression in primary care: defining and illustrating the policy context. J Clin Psychiatry. (1997) 58(Suppl 1):24–7.

PubMed Abstract | Google Scholar

12. Schoenbaum M, Unützer J, McCaffrey D, Duan N, Sherbourne C, Wells KB. The effects of primary care depression treatment on patients' clinical status and employment. Health Serv Res. (2002) 37:1145–58. doi: 10.1111/1475-6773.01086

PubMed Abstract | CrossRef Full Text | Google Scholar

13. Woltmann E, Grogan-Kaylor A, Perron B, Georges H, Kilbourne AM, Bauer MS. Comparative effectiveness of collaborative chronic care models for mental health conditions across primary, specialty, and behavioral health care settings: systematic review and meta-analysis. Am J Psychiatry. (2012) 169:790–804. doi: 10.1176/appi.ajp.2012.11111616

PubMed Abstract | CrossRef Full Text | Google Scholar

14. Lehti A, Hammarström A, Mattsson B. Recognition of depression in people of different cultures: a qualitative study. BMC Fam Pract. (2009) 10:53. doi: 10.1186/1471-2296-10-53

PubMed Abstract | CrossRef Full Text | Google Scholar

15. Kirmayer LJ, Robbins JM, Dworkind M, Yaffe MJ. Somatization and the recognition of depression and anxiety in primary care. Am J Psychiatry. (1993). 150:734–41. doi: 10.1176/ajp.150.5.734

PubMed Abstract | CrossRef Full Text | Google Scholar

16. Steinert C, Hofmann M, Kruse J, Leichsenring F. The prospective long-term course of adult depression in general practice and the community. A systematic literature review. J Affect Disord. (2014) 152, 65–75. doi: 10.1016/j.jad.2013.10.017

PubMed Abstract | CrossRef Full Text | Google Scholar

17. Bullinger M, Anderson R, Cella D, Aaronson N. Developing and evaluating cross-cultural instruments from minimum requirements to optimal models. Qual Life Res. (1993) 2:451–9. doi: 10.1007/BF00422219

PubMed Abstract | CrossRef Full Text | Google Scholar

18. Zhang J, Patel VL, Johnson TR, Shortliffe EH. A cognitive taxonomy of medical errors. J Biomed Inform. (2004) 37:193–204. doi: 10.1016/j.jbi.2004.04.004

PubMed Abstract | CrossRef Full Text | Google Scholar

19. Mitchell AJ. Clinical utility of screening for clinical depression and bipolar disorder. Curr Opin Psychiatry. (2012) 25:24–31. doi: 10.1097/YCO.0b013e32834de45b

PubMed Abstract | CrossRef Full Text | Google Scholar

20. Beaton DE, Bombardier C, Guillemin F, Ferraz MB. Guidelines for the process of cross-cultural adaptation of self-report measures. Spine. (2000) 25:3186–91. doi: 10.1097/00007632-200012150-00014

PubMed Abstract | CrossRef Full Text | Google Scholar

21. Brislin RW. Comparative research methodology: cross-cultural studies. Int J Psychol. (1976) 11:215–29. doi: 10.1080/00207597608247359

CrossRef Full Text | Google Scholar

22. Jones PS, Lee JW, Phillips LR, Zhang XE, Jaceldo KB. An adaptation of Brislin's translation model for cross-cultural research. Nurs Res. (2001) 50:300–4. doi: 10.1097/00006199-200109000-00008

PubMed Abstract | CrossRef Full Text | Google Scholar

23. Nabbe P, Le Reste JY, Guillou-Landreat M, Munoz Perez MA, Argyriadou S, Claveria A, et al. Which DSM validated tools for diagnosing depression are usable in primary care research? A systematic literature review. Eur Psychiatry. (2017) 39:99–105. doi: 10.1016/j.eurpsy.2016.08.004

PubMed Abstract | CrossRef Full Text | Google Scholar

24. Fitch K, Bernstein SJ, Aguilar MD, Burnand B, LaCalle JR, Lazaro P, et al. The RAND/UCLA Appropriateness Method User's Manual (2001).

Google Scholar

25. Fröjdh K, Håkansson A, Karlsson I, Frojdh K, Hakansson A. The Hopkins Symptom Checklist-25 is a sensitive case-finder of clinically important depressive states in elderly people in primary care. Int J Geriatr Psychiatry. (2004) 19:386–90. doi: 10.1002/gps.1102

PubMed Abstract | CrossRef Full Text | Google Scholar

26. Derogatis LR, Lipman RS, Rickels K, Uhlenhuth EH, Covi L. The Hopkins Symptom Checklist (HSCL): a self-report symptom inventory. Behav Sci. (1974) 19:1–15. doi: 10.1002/bs.3830190102

PubMed Abstract | CrossRef Full Text | Google Scholar

27. Nettelbladt P, Hansson L, Stefansson CG, Borgquist L, Nordström G. Test characteristics of the Hopkins Symptom Check List-25 (HSCL-25) in Sweden, using the Present State Examination (PSE-9) as a caseness criterion. Soc Psychiatry Psychiatr Epidemiol. (1993) 28:130–3. doi: 10.1007/BF00801743

PubMed Abstract | CrossRef Full Text | Google Scholar

28. Sandanger I, Moum T, Ingebrigtsen G, Dalgard OS, Sørensen T, Bruusgaard D. Concordance between symptom screening and diagnostic procedure: the Hopkins symptom checklist-25 and the composite international diagnostic interview I. Soc Psychiatry Psychiatr Epidemiol. (1998) 33:345–54. doi: 10.1007/s001270050064

PubMed Abstract | CrossRef Full Text | Google Scholar

29. Ventevogel P, De Vries G, Scholte WF, Shinwari NR, Faiz H, Nassery R, et al. Properties of the Hopkins symptom checklist-25 (HSCL-25) and the Self-Reporting Questionnaire (SRQ-20) as screening instruments used in primary care in Afghanistan. Soc Psychiatry Psychiatr Epidemiol. (2007) 42:328–35. doi: 10.1007/s00127-007-0161-8

PubMed Abstract | CrossRef Full Text | Google Scholar

30. Moum T. Mode of administration and interviewer effects in self-reported symptoms of anxiety and depression. Soc Indic Res. (1998) 45:279–318. doi: 10.1023/A:1006958100504

CrossRef Full Text | Google Scholar

31. Lipman RS, Covi' L, Shapiro AK. THE HOPKINS SYMPTOM CHECKLIST (HSCL) factors derived from the HSCL-90. J Affect Disord. (1979) 1:9–24. doi: 10.1016/0165-0327(79)90021-1

PubMed Abstract | CrossRef Full Text | Google Scholar

32. Spooner D, Pachana N. Ecological validity in neuropsychological assessment: a case for greater consideration in research with neurologically intact populations. Arch Clin Neuropsychol. (2006) 21:327–37. doi: 10.1016/j.acn.2006.04.004

PubMed Abstract | CrossRef Full Text | Google Scholar

33. Sousa VD, Rojjanasrirat W. Translation, adaptation and validation of instruments or scales for use in cross-cultural health care research: a clear and user-friendly guideline. J Eval Clin Pract. (2011) 17:268–74. doi: 10.1111/j.1365-2753.2010.01434.x

PubMed Abstract | CrossRef Full Text | Google Scholar

34. Maneesriwongul W, Dixon JK. Instrument translation process: a methods review. J Adv Nurs. (2004) 48:175–86. doi: 10.1111/j.1365-2648.2004.03185.x

PubMed Abstract | CrossRef Full Text | Google Scholar

35. Jones J, Hunter D. Consensus methods for medical and health services research. BMJ Clin Res Ed. (1995) 311:376–80. doi: 10.1136/bmj.311.7001.376

PubMed Abstract | CrossRef Full Text | Google Scholar

36. Haute Autorité Santé. Bases Méthodologiques Pour l'élaboration de Recommandations Professionnelles Par Consensus Formalisé. HAS (2006).

37. Linstone HA, Turoff M. The Delphi Method : Techniques And Applications. Advanced Book Program. Reading, MA: Addison-Wesley Pub. Co. (1975).

Google Scholar

38. Hasson F, Keeney S, McKenna H. Research guidelines for the delphi survey technique. J Adv Nurs. (2000) 32:1008–15. doi: 10.1046/j.1365-2648.2000.t01-1-01567.x

PubMed Abstract | CrossRef Full Text | Google Scholar

39. Graham B, Regehr G, Wright JG. Delphi as a method to establish consensus for diagnostic criteria. J Clin Epidemiol. (2003) 56:1150–6. doi: 10.1016/S0895-4356(03)00211-7

PubMed Abstract | CrossRef Full Text | Google Scholar

40. Hassan T, Barnett D. Delphi type methodology to develop consensus on the future design of EMS systems in the United Kingdom. Emerg Med J. (2002) 19:155–9. doi: 10.1136/emj.19.2.155

PubMed Abstract | CrossRef Full Text | Google Scholar

41. De Villiers MR, De Villiers PJT, Kent AP. The delphi technique in health sciences education research. Med Teach. (2005) 27:639–43. doi: 10.1080/13611260500069947

PubMed Abstract | CrossRef Full Text | Google Scholar

42. Bourrée F, Michel P, Salmi LR. Consensus methods: review of original methods and their main alternatives used in public health. Rev Epidemiol Sante Publiq. (2008) 56:e13–21. doi: 10.1016/j.respe.2008.10.005

PubMed Abstract | CrossRef Full Text | Google Scholar

43. Powell C. The Delphi technique: myths and realities. J Adv Nurs. (2003) 41:376–82. doi: 10.1046/j.1365-2648.2003.02537.x

PubMed Abstract | CrossRef Full Text | Google Scholar

44. Romm FJ, Hulka BS. Developing criteria for quality of assessment: effect of the Delphi technique. Health Serv Res. (1979) 14:309–12.

PubMed Abstract | Google Scholar

45. Jamieson S. Likert scales: how to (ab)use them. Med Educ. (2004) 38:1217–8. doi: 10.1111/j.1365-2929.2004.02012.x

PubMed Abstract | CrossRef Full Text | Google Scholar

46. Anadón M, Guillemette F. La recherche qualitative est-elle nécessairement inductive? Rech Qual. (2007) 5:26–37.

Google Scholar

47. Herdman M, Fox-Rushby J, Badia X. ”Equivalence” and the translation and adaptation of health-related quality of life questionnaires. Qual Life Res. (1997) 6:237–47.

PubMed Abstract | Google Scholar

48. Yu DSF, Lee DTF, Woo J. Issues and challenges of instrument translation. West J Nurs Res. (2004) 26:307–320. doi: 10.1177/0193945903260554

PubMed Abstract | CrossRef Full Text | Google Scholar

49. Daugherty JC, Puente AE, Fasfous AF, Hidalgo-Ruzzante N, Pérez-Garcia M. Diagnostic mistakes of culturally diverse individuals when using North American neuropsychological tests. Appl Neuropsychol. (2017) 24:16–22. doi: 10.1080/23279095.2015.1036992

PubMed Abstract | CrossRef Full Text | Google Scholar

50. Schnohr CW, Gobina I, Santos T, Mazur J, Alikasifuglu M, Välimaa R, et al. Semantics bias in cross-national comparative analyses: is it good or bad to have “fair” health? Health Qual Life Outcomes. (2016) 14:70. doi: 10.1186/s12955-016-0469-8

PubMed Abstract | CrossRef Full Text | Google Scholar

51. Hanrahan D, Sexton P, Hui K, Teitcher J, Sugarman J, London AJ, et al. Linguistic and cultural challenges in communication and translation in ussponsored HIV Prevention research in emerging economies. PLoS ONE. (2015). 10:e0133394. doi: 10.1371/journal.pone.0133394

PubMed Abstract | CrossRef Full Text | Google Scholar

52. Nabbe P, Le Reste JY, Guillou-Landreat M, Gatineau F, Le Floch B, Montier T, et al. The French version of the HSCL-25 has now been validated for use in primary care. PLoS ONE. (2019) 14:e0214804. doi: 10.1371/journal.pone.0214804

PubMed Abstract | CrossRef Full Text | Google Scholar

53. Skulmoski GJ, Hartman FT, Krahn J. The delphi method for graduate research. J Inf Technol Educ. (2007) 6:1. doi: 10.28945/199

CrossRef Full Text | Google Scholar

54. Vesga O, Agudelo M, Salazar BE, Rodriguez CA, Zuluaga AF. Generic vancomycin products fail in vivo despite being pharmaceutical equivalents of the innovator. Antimicrob Agents Chemother. (2010) 54:3271–9. doi: 10.1128/AAC.01044-09

PubMed Abstract | CrossRef Full Text | Google Scholar

55. Balliu C. L'enseignement de la traduction médicale: pour une nouvelle pragmatique. Meta J Traduct. (1994) 39:15–25. doi: 10.7202/001964ar

CrossRef Full Text | Google Scholar