Elsevier

Journal of Dentistry

Volume 95, April 2020, 103324
Journal of Dentistry

A methodological study to assess the measurement properties (reliability and validity) of a caries risk assessment tool for young children

https://doi.org/10.1016/j.jdent.2020.103324Get rights and content

Abstract

Objectives

To assess the CAMBRA caries risk assessment tool (CRAT) in terms of its measurement properties: reliability, validity and responsiveness to change in caries estimates over time.

Methods

Secondary data from the VicGeneration (VicGen) birth cohort study were used. Caries risk status at child age18 months was compared against caries development at 36 and 48 months. Questionnaire data for the 18-month-old children, were used to generate the CRAT item responses. Five examiners assessed the caries risk profile for each 18-month-old child and assigned a risk status. The analytic method was informed by the COnsensus-based Standards for the selection of health Measurement INstruments Checklist.

Results

Each examiner applied the CAMBRA CRAT to 214 cases from the VicGen study. At 18 months of age, the proportion of low-, moderate- and high-risk children were 17 %, 17 % and 66 %, respectively. Caries prevalence at 36 and 48 months of age was 18 % and 35 %, respectively. Almost perfect reliability (both inter- and intra-reliability) was achieved in caries risk categorisation. For criterion validity and responsiveness, the area under the curve was approximately 0.50. For the dichotomised risk, the sensitivity of the CAMBRA CRAT to predict caries at 36 and 48 months, was 74 % and 70 %, respectively. The corresponding specificity scores were 35 % and 36 %, respectively.

Conclusions

The results from this study show that the CAMBRA CRAT, in its current form, is not a good predictor of future caries among young Victorian children and in fact overestimates the proportion of children at high-risk of future disease. As such, it may not be ideal for use among young Victorian children and it is highly likely it’s use could be driving over treatment.

Clinical Significance

: This study provides evidence for the applicability of the CAMBRA CRAT with young Victorian children to inform decision making on caries risk-based management. It also highlights the important measurement properties and related analytics (applicable to any CRAT) that should be considered when selecting a CRAT for use in clinical practice.

Introduction

Dental caries in children is common. In the deciduous dentition the disease can manifest as early as the first year of life and continue until exfoliation [1]. The factors contributing to caries risk in the early years (0–5 years) have been shown to differ from those in older children [2]. This is understandable given the rapid changes in diet, tooth development and oral hygiene practices as a child progresses through developmental stages from infant to preschool age. The well substantiated influences on caries in young children are: parenting behaviours, knowledge, beliefs and attitudes; infant feeding practices; maternal circumstances and oral health; and infant-related oral health behaviours [2,3]. Understanding the age specific influences on child oral health and using this knowledge to inform the design and implementation of interventions (including service delivery), has greater potential to translate to effective action, compared with blanket (broad spectrum) interventions to improve oral health or service delivery across all ages.

A recent study showed that caries prevalence, among children in Victoria, Australia, increased from 8 % to 23 % between 18 and 36 months of age, with the development of new lesions on previously sound tooth surfaces [4]. It is evident from the study by Gussy et al. [4], that the time period between 18 and 36 months is a window of opportunity for early intervention and disease prevention. However, to act effectively during this window of opportunity, at-risk children need to be accurately and precisely identified. Valid and reliable tools that discriminate children truly at risk of dental caries from those not at risk are required.

Risk-based management using caries risk assessment tools (CRATs) is promoted as a way of: identifying those at highest risk of disease, guiding treatment decisions, guiding discussions with patients (including parents/caregivers) regarding caries risk/protective factors, determining appropriate recalls and to inform the economics of producing, delivering and using health care [5]. However, whether risk-based management is achieving these outcomes for either the patient or the health service organization is unclear. From the information reported in two recent systematic reviews of the measurement properties (reliability and validity) of CRATs [6,7], it can be extrapolated that current CRATs used in risk-based management may in fact result in overtreatment and as such are wasteful of resources. This has major implications for public/community dental services with limited resources and where risk-based management is used as a mechanism to ensure appropriate resource allocation, so that people with the most need get the required care. Hence, research is urgently required to provide the evidence to demonstrate that risk-based management and CRATs are an efficient and effective method to improve oral health and health service outcomes.

Numerous caries risk assessment tools (CRATs) are currently being used in clinical dental practice with the popular ones being: Caries Management by Risk Assessment (CAMBRA), American Academy of Pediatric Dentistry CRAT and Cariogram [[8], [9], [10]]. These CRATs are a collation of various caries risk assessment methods (CRAMs), that have been compiled into prediction models, with the assumption that together the CRAMs will be able to predict future disease better than any single method alone. As stated previously, the evidence that CRATs accurately predict future disease is unclear and in fact may be resulting in overtreatment and wastage of resources. Commonly used CRAMs include socio-economic status, dental service use, diet and oral health behaviours.

The ability of a CRAT to perform well by correctly and consistently identifying those individuals at risk of future disease is affected by the quality of two core measurement properties – reliability and validity. Reliability is the property of the CRAT to produce similar results under different conditions, whereas validity relates to the degree to which the CRAT actually measures what it was designed to measure. Reliability includes the domains of internal consistency and measurement error. Validity includes the domains - content, construct, and criterion validity. Responsiveness, a relatively new addition to these classic measurement properties, is the ability of an instrument to detect change in the construct over time [11].

Evidence for the quality of measurement properties of CRATs is required to inform tool selection for use in practice, however, current research on the topic is very limited [7,12]. To investigate this topic in-depth, we conducted a comprehensive systematic review on the quality of measurement properties for current CRATs for young children [6]. This review identified a major knowledge gap by demonstrating the lack of methodological studies to provide the evidence to assess the quality of the measurement properties for the identified CRATs.

The authors of this study aimed to address the knowledge gap by conducting a methodological study to assess the measurement properties for the CAMBRA CRAT. Specifically, the objectives were to assess the CAMBRA CRAT in terms of its: reliability in application, validity in predicting future caries in young children, and responsiveness to change in caries estimates over time. The CAMBRA CRAT for children under 6 years of age was selected for use in this study, as it showed strong evidence for content validity [6]; and although not the best performing CRAT in our recent review, is currently (in its original or modified versions) the most commonly used CRAT across many public dental services in Victoria, Australia. The CAMBRA CRAT was used as a case study and the method demonstrated in this study could be used as a road map for testing other CRATs.

Section snippets

Materials and methods

The University of Melbourne Human Research Ethics Committee approved this study Ethics ID 1851421.

The CAMBRA CRAT was assessed using a secondary analysis of data from the VicGeneration (VicGen) study, an oral health birth cohort study to understand the multifactorial nature of early childhood caries development [13]. Seven waves of data collection occurred in VicGen, at child ages (months) 1, 6, 12, 18, 36, 48 and 60; and each wave included a clinical dental examination, saliva collection and a

Sample size considerations

To achieve an excellent rating for the quality of a measurement property, the COSMIN checklist recommends a sample size of at least 100 cases. This study included 214 cases which had complete data across the three time points – 18, 36 and 48 months of age.

CAMBRA CRAT items and VicGen variables

As the CAMBRA CRAT items did not have specific definitions, the author team defined, jointly discussed and agreed on the VicGen variables that best represented the CAMBRA CRAT items. In some cases, two or more VicGen variables were aligned with the CRAT item and in such cases composite variables of the two (or more) variables were generated. For CRAT items where there was no clearly aligned VicGen variable, logic and evidence-informed assumptions were used to derive the appropriate variables.

Data cleaning and management

STATA was used to clean and manage the VicGen data, and in creating the analysis datasets. Four separate datasets were used to create the master dataset – the 18-month questionnaire data and the clinical data for 18, 36 and 48 months of age. The master dataset was then used to derive the variables for the CAMBRA CRAT items based on the 18-month questionnaire data. Once the CAMBRA variables were generated, another dataset with only the child unique identification numbers and the CAMBRA CRAT

Caries risk assessments

Five examiners (four examinations as BC + HC jointly examined to establish the reference standard) with a background in clinical dentistry were trained to apply the CAMBRA CRAT using the online CAMBRA Basic training course available at https://course.mycambra.com/courses/cambra-basic/. The caries risk assessments were conducted in REDCap, where the examiner was required to determine whether a child was high-, moderate- or low-risk based on yes/no responses to the CAMBRA CRAT items (which were

Data analysis

On completion of the caries risk assessment stage of the research, the data were exported from REDCap into STATA (version 15.1, StataCorp, College Station, TX, USA) for analyses. Eighteen-month-old children who had a clinical examination at 36 and 48 months of age were included in this analysis (n = 214). All children were caries free at 18-months of age. In this analysis, caries/caries lesion refers to any clinically apparent sign of tooth demineralisation [non-cavitated and cavitated lesions

Results

Five examiners applied the CAMBRA CRAT to 214 cases from the VicGen birth cohort study. At 18 months of age, the proportion of children categorised as low-, moderate- and high-risk were 17 %, 17 % and 66 %, respectively. The prevalence of caries at 36 and 48 months of age was 18 % and 35 %, respectively. Results of the sub-analysis showed that the CRAT items responsible for the higher proportion of high-risk categorisations were related to the child’s dental service utilisation - ‘Child: no

Discussion

The results of this study show that the CAMBRA CRAT in its current form may not be useful in this population. An obvious observation from the analysis was the higher proportion of children (66 %) designated as high-risk at baseline (18-months of age) compared with the proportion that developed caries lesions at 36 (18 %) and 48 (35 %) months of age. Gao et al. [18], made a similar observation in their validation study and reported that the majority of children were considered high-risk when the

Conclusions

This study provides the evidence to show that the CAMBRA CRAT, in its current form, may not be ideal for use in risk-based disease management among young Victorian children. Due to its low specificity scores it is highly likely that the use of this CRAT in Victoria could be driving over treatment, which is a major issue particularly for public dental services where risk-based service delivery is used as a resource allocation and waitlist management method, to prioritise care for those most at

CRediT authorship contribution statement

Bradley Christian: Conceptualization, Methodology, Software, Data curation, Writing - original draft, Writing - review & editing, Project administration, Formal analysis. Hanny Calache: Conceptualization, Methodology, Writing - original draft, Writing - review & editing, Supervision. Geoff Adams: Conceptualization, Methodology, Data curation, Writing - original draft, Writing - review & editing, Supervision, Formal analysis. Martin Hall: Conceptualization, Methodology, Writing - original draft,

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Acknowledgements

Dr. Ghulam Murtaza, Digital Research Analyst, La Trobe University, for support with REDCap. Ms. Lauren Carpenter for advice on the VicGen data. The clinical examiners for providing their valuable time for this study.

References (32)

  • S. Twetman et al.

    Risk assessment - can we achieve consensus? Community Dent

    Oral Epidemiol.

    (2013)
  • B. Christian et al.

    A systematic review to assess the methodological quality of studies on measurement properties for caries risk assessment tools for young children

    Int. J. Paediatr. Dent.

    (2019)
  • M. Tellez et al.

    Evidence on existing caries risk assessment systems: are they predictive of future caries? Community Dent

    Oral Epidemiol.

    (2013)
  • AAPD

    Guideline on caries-risk assessment and management for infants, children, and adolescents

    Pediatr. Dent.

    (2014)
  • D. Bratthall et al.

    Cariogram--a multifactorial risk assessment model for a multifactorial disease, Community Dent

    Oral Epidemiol.

    (2005)
  • J.D. Featherstone et al.

    Caries management by risk assessment: consensus statement, April 2002

    J. Calif. Dent. Assoc.

    (2003)
  • Cited by (0)

    View full text