Application of a four-dimensional framework to evaluate the quality of the HIV/AIDS data collection process in China

https://doi.org/10.1016/j.ijmedinf.2020.104306Get rights and content

Abstract

Objective

To qualitatively evaluate the quality of the data collection process used by the Chinese national HIV/AIDS data repository (CRIMS), using a four-dimensional (4D) framework. The process is vital for the acquisition of high-quality data for ending the HIV/AIDS epidemic in China.

Methods

The study was carried out in China from September 2014 to April 2015. Stratified convenient sampling was conducted to recruit 28 study participants including health administrators, public health professionals and clinicians. Data were collected through semi-structured interviews with the participants and from field observations in six hospitals. Content analysis was conducted following the 4D Framework.

Results

61 percent of the facilitators and 74 percent of the barriers of the 4D Framework were confirmed in the CRIMS data collection process. The CRIMS achieved better-quality data collection management. The perceived gaps primarily included: impractical data collection protocol and invalid quality assessment mechanism for data collection management; weak leadership and unsupportive organizational policy for data collection environment; poor communication and job fatigue for data collection personnel; and inflexibility and inaccessibility of data collection system. Areas for improvement included: engaging frontline staff in the design of data collection protocol, standardizing quality assurance procedures, strengthening leadership, recognizing data collector’s contributions, and meeting end-users’ needs for the CRIMS.

Conclusion

The findings generated knowledge about the quality of the CRIMS data collection process. The 4D Framework has potential as an evaluation tool for decision-makers on the improvement of the public health data collection process.

Introduction

Data is vital for public health program decision making and intervention, for example in prevention and control of the epidemic of human immunodeficiency virus (HIV) and acquired immune deficiency syndrome (AIDS) [[1], [2], [3]]. To achieve the goal of ending the HIV/AIDS epidemic by 2030, the Joint United Nations Programme on HIV/AIDS (UNAIDS) recommends that member countries collect, analyse and disseminate high-quality HIV/AIDS data [4]. Since HIV/AIDS data is captured in national public health information systems (PHISs) [5,6], the quality of the PHIS data collection process is vital for acquisition of high-quality HIV/AIDS data.

The HIV/AIDS epidemic has remained a critical public health challenge in China [3,7,8]. By October 2019, about 958,000 persons were recorded living with HIV/AIDS (PLWHA) nationwide. The 2018 national HIV/AIDS epidemic estimation results indicate the actual number of PLWHA ranged from 1.1 to 1.4 million by the end of 2018 and keep growing in the near future [8,9]. The Chinese HIV/AIDS Comprehensive Response Information Management System (CRIMS) is a national repository of data for HIV/AIDS “project planning, budgeting, implementation, monitoring and evaluation” [6]. The CRIMS data collection process needs to be of high quality to meet the information needs of the decision-makers on HIV/AIDS prevention and control.

The CRIMS commenced officially in 2008 as a sub-system of the China Information System for Disease Control and Prevention, which is a large-scale web-based disease surveillance system [5,7]. A variety of electronic reporting forms have been developed for data collection and entry into the CRIMS by the Chinese Centre for Disease Control and Prevention (China CDC) [5,6,10]. The data sources for these forms primarily include case reporting and management, healthcare services for PLWHA, intervention services on high-risk groups, and national HIV/AIDS prevention and control program management [5,11]. The major data producers and collectors for the system are the county CDCs and hospitals that provide public health services and interventions to the target groups [11].

In the last decade, a data-driven performance assessment scheme has been established for assessing the data quality of the CRIMS [5,12]. Implementation of the scheme has led to an improvement in the quality of the reporting data in the system [12,13]. However, certain reporting data, e.g., case demographics, case follow-up, and intervention delivery, were still inaccurate, incomplete, missing, delayed, under-reported or leaking [[14], [15], [16], [17]]. The national data quality assessment of intervention in the population at high risk for HIV/AIDS between 2014 and 2018 suggested that 79.4 % (70.5 %–98.3 %) of the data recorded in the CRIMS and in the paper records were consistent. However, in 2018 four types of consistency rates dropped to 85.3 %, 91.0 %, 78.8 %, 70.5 %, respectively, all ranking lowest within the five-year span [18,19]. A literature review of the CRIMS data management studies in peer reviewed Chinese and English electronic databases showed that 61 % (37/61) of the studies focused on assessing the quality of data stored in the CRIMS [13]. The other studies focused on the development and management of the information systems or the influential factors on data collectors. Few studies identified or provided evidence on where, when and how data quality problems occur, the causes of poor data quality, or what strategies can be implemented to improve data quality. As data quality problems often occur in the data collection process [[18], [19], [20], [21]], there is an urgent need to understand the factors influencing the performance of the process so as to generate insight on data quality management for the CRIMS.

To date, the quality of the PHIS data collection process is an under-researched area [22]. Our previous study identified that several PHIS data quality assessment methods were focused on the data collection procedure, i.e., data recording, storage and audits, and the functions of the PHIS system that facilitate or hinder data collection [[22], [23], [24]]. Little attention had been given to the effect of the contextual factors (organizational, personnel or environmental) on the quality of the data collection process [22]. To address this knowledge gap, we have constructed a four-dimensional (4D) framework based on a systematic literature review of the topic from the international literature [25]. Unlike other data quality frameworks issued by public health institutions such as the United States CDC’s Guidelines for Evaluating Public Health Surveillance Systems and the CIHI Data Quality Framework [20,23,24], the 4D Framework is primarily focused on assessing the PHIS data collection process.

An expert elicitation study to validate the structure of the 4D Framework confirmed that it should cover four dimensions (data collection management, data collection environment, data collection personnel, and data collection system) [26]. These dimensions comprised 16 subcomponents and 116 indicators including 82 facilitators and 34 barriers (see Fig. 1). By providing guidance to practitioners to harness the facilitators and to address the barriers, the 4D Framework can be a promising quality improvement model to strengthen the management of the public health data collection process.

In this study we applied the 4D Framework to assess the quality of the CRIMS data collection process. We aimed to identify the gaps in the process and suggest improvement strategies for HIV/AIDS data collection in China.

Section snippets

Study design and procedure

We carried out a qualitative research study in China from September 2014 to April 2015. A semi-structured interview guide was developed to focus on identification of what, where, when and how data quality problems occur in the CRIMS data collection process. The interview guide covered the perceptions, cognition, and experiences of the study participants about the HIV/AIDS data collection and quality management (see Appendix Table A). The questions included “What is your experience with the

Quality of the CRIMS data collection process

Of the 116 indicator statements in the 4D Framework, 75 (65 %) were confirmed in this research. Of these, 50 were facilitators (accounting for 61 % of the 82 facilitators) and 25 were barriers (74 % of the 34 barriers) (see Table 2).

The proportion of facilitators confirmed for Data Collection Environment was low at 37 % (10/27); with the other dimensions the proportions were 68 % (19/28) for Data Collection Management, 94 % (12/17) for Data Collection Personnel and 90 % (9/10) for Data

Discussion

This study evaluated the quality of the CRIMS data collection process for HIV/AIDS prevention and control in China. Through interviewing the domain experts who were directly involved with data collection and observing the clinical HIV/AIDS data collection process, the study confirmed 61 % of the facilitators and 74 % of the barriers from the original 4D Framework. The findings might have identified the ‘real’ factors behind the ‘dropping’ consistency rates of 2018 in the CRIMS data collection

Conclusion

This study analyzed and synthesized the China CDC HIV/AIDS data collection management practices across all levels of healthcare organizations engaged in this process. It identified the process gaps and mapped these to the indicator statements in the 4D Framework. It also provided strategies to address these gaps. The findings support a multi-dimensional approach to improve performance of the PHIS data collection process by the top level of the country’s health system instead of by individual

Authors’ contributions

HC designed and conducted the interviews, made data analysis and writing of the manuscript. PY conceptualized and designed the study, reviewed the data and writing of the manuscript. DH and TC reviewed the data and made critical revisions to the paper. All authors reviewed the data and contributed to their interpretation.

Ethical approval

Ethical approval was granted by the Human Research Ethics Committee at the University of Wollongong and the Institutional Review Board of National Centre for AIDS/STD Control and Prevention at China CDC.

Funding

This research has been conducted with the support of an Australian Government Research Training Program Scholarship.

Summary Points

What was already known on the topic?

  • High-quality data collection process is essential for Chinese HIV/AIDS prevention and control, but the quality of the process is not known.

  • A 4D Framework has been used for defining the quality of the data collection process in public health information systems through considering data collection management, data collection

Declaration of Competing Interest

The authors report no declarations of interest.

References (34)

  • H.I. Hall

    Describing the HIV/AIDS epidemic: using HIV case data in addition to AIDS case reporting

    Ann. Epidemiol.

    (2005)
  • N. Wang

    Estimating the number of people living with HIV/AIDS in China: 2003-09

    Int. J. Epidemiol.

    (2010)
  • Z. Wu

    HIV/AIDS in China: Beyond the Numbers

    (2017)
  • Joint United Nations Programme on HIV/AIDS (UNAIDS). Data

    (2019)
  • Y. Mao

    Development of a unified web-based national HIV/AIDS information system in China

    Int. J. Epidemiol.

    (2010)
  • Z. Wu

    The integration of multiple HIV/AIDS projects into a coordinated national programme in China. World Health Organization

    Bull. World Health Organ.

    (2011)
  • Z. Liu

    Changing epidemiological patterns of HIV and AIDS in China in the post-SARS era identified by the nationwide surveillance system

    BMC Infect. Dis.

    (2018)
  • F. Lyu et al.

    National HIV/AIDS epidemic estimation and interpretation in China

    Chin. J. Epidemiol.

    (2019)
  • N. He

    Emerging changes and characteristics of the HIV epidemic in China

    Shanghai J. Prevent. Med.

    (2019)
  • Q. Qin

    Quality analysis on the direct report network for national HIV/AIDS in 2006

    Dis. Surveill.

    (2007)
  • J. Xu et al.

    Current status and prospective of Chinese AIDS CRIMS

    Chin. J. AIDS STD

    (2016)
  • Y. Liu

    Quantitatively monitoring AIDS policy implementation in China

    Int. J. Epidemiol.

    (2010)
  • H. Chen
    (2017)
  • J. Han

    Analysis of national data quality assessment for follow up of HIV/AIDS cases between 2009 and 2013

    Chin. J. Prev. Med.

    (2014)
  • Q. Qin

    Evaluation of AIDS network direct reporting quality and related indicators in China, 2010

    Dis. Surveill.

    (2012)
  • X.-q. Hui

    Evaluation on the data quality of HIV screening project in AIDS demonstration area of Henan Province

    Chin. J. Dis. Control Prev.

    (2016)
  • Y. He et al.

    Security of 275 AIDS patient information leaking

    China Econ. Weekly

    (2016)
  • Cited by (1)

    View full text