Elsevier

Annals of Epidemiology

Volume 24, Issue 1, January 2014, Pages 63-71
Annals of Epidemiology

Original article
Influence of population versus convenience sampling on sample characteristics in studies of cognitive aging

https://doi.org/10.1016/j.annepidem.2013.10.005Get rights and content

Abstract

Purpose

We examined whether differences in findings of studies examining mild cognitive impairment (MCI) were associated with recruitment methods by comparing sample characteristics in two contemporaneous Australian studies, using population-based and convenience sampling.

Method

The Sydney Memory and Aging Study invited participants randomly from the electoral roll in defined geographic areas in Sydney. The Australian Imaging, Biomarkers and Lifestyle Study of Ageing recruited cognitively normal (CN) individuals via media appeals and MCI participants via referrals from clinicians in Melbourne and Perth. Demographic and cognitive variables were harmonized, and similar diagnostic criteria were applied to both samples retrospectively.

Results

CN participants recruited via convenience sampling were younger, better educated, more likely to be married and have a family history of dementia, and performed better cognitively than those recruited via population-based sampling. MCI participants recruited via population-based sampling had better memory performance and were less likely to carry the apolipoprotein E ε4 allele than clinically referred participants but did not differ on other demographic variables.

Conclusion

A convenience sample of normal controls is likely to be younger and better functioning and that of an MCI group likely to perform worse than a purportedly random sample. Sampling bias should be considered when interpreting findings.

Introduction

Epidemiologic studies differ regarding findings about rates of decline and prognosis of mild cognitive impairment (MCI), an intermediate state between normal aging and dementia. Differences in study findings could be associated with differences in sampling methods. Studies may use population-based sampling which aims to select a random group of participants who are representative of the population of interest or convenience sampling, which involves engaging volunteers who are selected due to ease of recruitment and willingness to participate and clinical referrals who are selected to maximize the sampling of specific types of disorders.

Convenience sampling of cognitively normal (CN) participants is vulnerable to self-selection bias as those who seek out opportunities to participate in cognitive research may be more capable and motivated than randomly recruited CN participants. Consistent with this, studies have shown that CN convenience samples tend to be younger [1], [2], [3] and better educated [1], [2], [3], [4] than those recruited via population-based sampling and more likely to have a family history of Alzheimer disease (AD) [3], probably reflecting their personal interest and motivation.

Clinically referred samples are also susceptible to bias as they may contain people who have better access to health care due to socioeconomic factors or have more complex or severe conditions [5]. Consistent with such a bias, clinically referred MCI participants tend to be better educated [3], [6], [7], [8], [9] and more likely to be married and living independently than people with MCI in the wider population [6], [9]. They also tend to be younger, possibly because doctors are more likely to refer younger patients to specialty clinics [3], [6], [7], [10], [11], although some studies have found them to be older [8], [12]. Additionally, clinically recruited MCI and AD participants are more likely to carry the apolipoprotein E (APOE) ε4 allele [3], [7] and more likely to decline faster suggesting more aggressive brain pathology [3].

Such demographic differences between population-based and convenience samples could lead to invalid research conclusions. For example, younger age of convenience samples could affect the validity of research examining neuropathology of MCI, effects of anti-AD medications, and APOE genotype [7], [11]. Similarly, higher levels of education observed in convenience samples may be associated with greater levels of cognitive reserve and could lead to incorrect conclusions regarding MCI progression rates.

There are mixed findings as to whether sampling methods are associated with differences in cognitive performance of CN samples. CN convenience samples outperformed population-based participants on the Mini-Mental State Examination (MMSE) [2], [3] and on a vocabulary task possibly due to higher education levels [4] but not on reasoning or word recall tasks [4].

Similarly, there is mixed evidence as to whether sampling methods are associated with differences in cognitive performance of MCI samples. There is some evidence that population-based MCI samples outperform clinic samples (solely based on MMSE) possibly because participants from clinics have a more aggressive or advanced form of MCI [6], [7], [12]. By contrast, others found no difference between clinic and population samples on the MMSE, memory tasks, or executive function tasks [8] or found that clinic samples performed better possibly due to higher levels of cognitive reserve although this result was not corrected for differences in sample age and education [3].

Potential cognitive differences merit further investigation. If there is consistent evidence that CN convenience samples outperform population-based samples, then studies comparing MCI participants against a convenience sampled normal reference group would exaggerate their degree of cognitive impairment. Additionally, evidence indicating that clinically referred MCI samples cognitively underperform compared to population-based MCI samples, suggests that clinic samples consist of a select group of patients with a form of MCI are more likely to progress to dementia and do not represent the heterogeneity of MCI in the general population.

Additionally, as convenience sampling is more selective than population-based sampling, one may expect less interindividual variability among convenience samples. In one study, convenience samples showed less variance than population-based samples in some quality of life and social relationship variables but not cognitive measures [4].

This study examined the relationship between recruitment method and demographic and cognitive characteristics of the purportedly random electoral roll–based sample used in the Sydney Memory and Aging Study (MAS) [13] and a convenience sample of CN participants recruited via media advertisement and clinical referrals with MCI used in the Australian Imaging and Biomarkers Lifestyle (AIBL) Study of Ageing [14]. We hypothesized that CN and MCI participants in the MAS sample would be older, less educated, less likely to be married, and less likely to be living independently than those in the AIBL study. Additionally, we hypothesized that the AIBL study would contain more CN participants with a family history of memory problems or dementia and more MCI participants who were APOE ε4 carriers than the MAS. There were no clear predictions regarding differences between the samples on cognitive performance or on interindividual variability on cognitive measures.

Section snippets

Protocols

Baseline data were obtained from two Australian longitudinal studies of cognitive aging: the MAS and the AIBL study. The MAS [13] was initiated in 2005 and conducted in Sydney. Participants were recruited from the community via the electoral roll (in Australia, voting is compulsory). A random sample of 8914 people living in the federal government electorates of Kingsford-Smith and Wentworth aged between 70 and 90 years were invited by letter to participate. Of these, 1772 people (20%) agreed to

CN comparisons

Table 2 shows comparisons between MAS and AIBL study CN samples. The MAS sample was older and less educated than the AIBL study sample and had fewer participants who were married or in de facto relationships and more participants who were widowed or had never been married. The samples did not differ in sex ratios, living arrangements, or the percentage of APOE ε4 carriers. Fewer MAS participants had a family history of dementia/memory problems.

The AIBL study sample outperformed the MAS sample

Discussion

We compared MAS and AIBL study samples to examine whether differences in their recruitment methods were associated with differences in the demographic and cognitive characteristics of their samples. The hypothesis that the MAS sample would be older, less educated, less likely to be married, and less likely to be living independently than the AIBL study sample was supported among CN participants but not among MCI participants. The hypothesis that AIBL study CN participants would be more likely

Acknowledgments

The authors thank all participants and their informants for their enthusiastic support. The authors also thank Kristan Kang, Joanne Robertson, Lance Macaulay, and the MAS and AIBL study research teams. This study was supported by Commonwealth Scientific and Industrial Research Organisation under the Preventative Health Flagship. The MAS is supported through National Health and Medical Research Council Program grant (ID 568969). The AIBL study receives support from the Science Industry Endowment

References (50)

  • R. Barnhart et al.

    Geographically overlapping Alzheimer’s disease registries: comparisons and implications

    J Geriatr Psychiatry Neurol

    (1995)
  • D. Tsuang et al.

    Impact of sample selection on APOE epsilon4 allele frequency: a comparison of two Alzheimer's disease samples

    J Am Geriatr Soc

    (1996)
  • S. Farias et al.

    Progression of mild cognitive impairment to dementia in clinic- vs community-based cohorts

    Arch Neurol

    (2009)
  • J. Schneider et al.

    The neuropathology of older persons with and without dementia from community versus clinic cohorts

    J. Alzheimers Dis

    (2009)
  • F. Andersen et al.

    Recruitment methods in Alzheimer's disease research: general practice versus population based screening by mail

    BMC Med Res Methodol

    (2010)
  • P. Sachdev et al.

    The Sydney Memory and Ageing Study (MAS): methodology and baseline medical and neuropsychiatric characteristics of an elderly epidemiological non-demented cohort of Australians aged 70-90 years

    Int Psychogeriatr

    (2010)
  • K. Ellis et al.

    The Australian Imaging, Biomarkers and Lifestyle (AIBL) study of aging: methodology and baseline characteristics of 1112 individuals recruited for a longitudinal study of Alzheimer’s disease

    Int Psychogeriatr

    (2009)
  • R. Petersen

    Challenges of epidemiological studies of mild cognitive impairment

    Alzheimer Dis Assoc Disord

    (2004)
  • H. Nelson et al.

    National adult reading test (NART): test manual

    (1991)
  • D. Wechsler

    Wechsler test of adult reading: examiner’s manual

    (2001)
  • D. Wechsler

    Wechsler adult intelligence scale-III

    (1997)
  • A. Rey

    L’examen clinique en psychologie

    (1964)
  • M. Harris et al.

    Mayo's Older Americans Normative Studies: expanded AVLT Recognition Trial norms for ages 57 to 98

    J Clin Exp Neuropsyc

    (2002)
  • R. Ivnik et al.

    The Auditory-Verbal Learning Test (AVLT): norms for ages 55 years and older

    Psychological Assessment

    (1990)
  • R. Ivnik

    Mayo's Older Americans Normative Studies: updated AVLT norms for ages 56 to 97

    Clinical Neuropsychology

    (1992)
  • Cited by (57)

    • Building knowledge ambidexterity using cloud computing: Longitudinal case studies of SMEs experiences

      2022, International Journal of Information Management
      Citation Excerpt :

      We adopted convenience sampling as it is a type of nonprobability and non-random sampling where members of the target population who meet certain practical criteria are included in the study (Etikan et al. 2016). Additionally, convenience sampling is more selective than other sampling techniques as variability among sample groups will be less (Brodaty et al. 2014). The research sample group consisted of four SMEs based in India representing both small-sized and medium-sized businesses that operate in various industries with considerable experience and expertise in their respective sectors (see Table 2).

    View all citing articles on Scopus
    View full text