Introduction

Rare genetic conditions are a leading cause of paediatric morbidity and mortality, and account for a substantial number of paediatric hospital admissions [1,2,3]. The life-threatening and time-critical circumstances necessitate a timely aetiologic diagnosis but the heterogeneous, often overlapping or atypical, clinical presentations pose a substantial challenge [4]. Rapid genomic testing has been shown to enable a timely molecular diagnosis within 3–23 days, with diagnostic, clinical, family, and economic benefits [5,6,7,8,9,10,11,12,13,14,15,16,17]. Rapid genomic testing in neonatal and paediatric intensive care has been heralded as genomic medicine’s ‘critical application’ [18] and is on the threshold of being integrated into clinical pathways and scaled up from single hospitals to national healthcare systems [9].

Implementation of rapid genomic sequencing programmes, however, requires significant investment, and a real trade-off between reducing the time to test results and the cost of delivering genomic testing exists. To optimise the implementation of rapid genomic testing in paediatric critical care, it is pertinent to identify and value the associated diagnostic, clinical, non-clinical, and process outcomes provided to children and families [19]. This information is necessary to enable decisions that maximise the benefit to society with given healthcare resources. Yet, no information exists on the value that people may place on the benefits of rapid genomic testing. Perhaps the most promising approach to valuing these benefits is the discrete choice experiment (DCE) [20,21,22], and the application of DCEs to elicit preferences and values for genomic testing for informing decision-making priorities is increasing [23,24,25,26].

We conducted a DCE to estimate the value that society and people with lived experiences of rapid genomic testing place on genomic testing for critically ill infants and children with a suspected genetic condition. As healthcare systems increasingly consider widespread implementation of rapid genomic testing of critically ill children, the findings of our work can be used in a cost–benefit analysis to determine whether it provides value for the available public healthcare resources. The findings are also important in terms of understanding how preferences for rapid genomic testing may vary depending on individual personal characteristics as well as illustrating potential differences in preferences and values between society and families with a lived experience of a critically ill child.

Methods

Study design and participants

We developed a DCE survey to elicit preferences and values for genomic testing in paediatric critical care in line with good research practice recommendations [27,28,29]. DCEs essentially comprise two phases. In the first phase, the key attributes (characteristics) and associated levels of a healthcare technology or service are identified, preferably using qualitative methods [30,31,32]. The second phase involves experimental design methods to construct a series of choice tasks [28], whereby respondents are asked to choose between two or more alternative options that involve different combinations of attribute levels. Econometric analysis of choice data is then used to reveal the underlying preferences and value, expressed as willingness to pay (WTP) [33]. An illustration of the key steps involved in our DCE is provided in Fig. 1.

Fig. 1: Graphical representation of the key steps involved in our discrete choice experiment.
figure 1

Qualitative methods (step 1) are used for the development and labelling of attributes. The qualitative phase subsequently informs the development of the survey and the choices presented to participants, which are selected using statistical methods (step 2 - quantitative phase 1). The final step (step 3 - quantitative phase 2), involves the analysis of the survey data.

Full size image

Focus groups were conducted with key stakeholders (community representatives or advocates with lived genomic experience, operational genomic staff, and clinical genomic staff) with the purpose of identifying the attributes that reflect different aspects of genomic testing information that are important in the context of critically ill children. Detailed information about the focus groups is provided elsewhere [19].

Table 1 presents the five attributes and attribute levels that were included in the DCE. The attributes were: (1) ‘Number of children who receive genetic diagnosis’, (2) ‘time between test initiation and results’, (3) ‘chance of improving the process of child’s medical care’, (4) ‘chance of improving child’s health outcomes’, and (5) ‘cost of testing to you’. Attribute labelling was defined using an iterative approach with the stakeholders involved in the focus groups [32]. Attribute levels were finalised in consultation with the genetic experts of our research team to ensure clinical face validity.

Table 1 Attributes and attribute levels.
Full size table

A Bayesian D-efficient and explicit partial profiles experimental design with 48 choice tasks, split into four blocks (i.e. 12 choice tasks per respondent), was developed using Ngene [34]. The benefits of Bayesian efficient designs in terms of the robustness of design and reliability of estimated parameters are described elsewhere [27, 28, 35]. Explicit partial profiles were used because they reduce the complexity of choice-making by holding one or more attributes fixed between alternatives within a given choice task [34, 36]. Participants therefore have less trade-offs to consider when making choices between alternatives. This is an attractive feature, particularly in complex contexts like paediatric critical care. In our design, choice tasks consisted of two overlapping and three varying attributes across alternatives. Each choice task had different overlapping attributes or attribute levels. Choice tasks were selected from a candidature set of all relevant attribute combinations using the modified Fedorov algorithm [34]. Restrictions to the algorithm were applied to ensure representation of all attribute levels within the DCE. Choice tasks asked participants to indicate the situation under which they would choose for their child to have a genomic test. Participants could choose between three options (Situation 1, Situation 2, or the opt out ‘I would not like my child to have a genomic test’). An example of a choice task is shown in Fig. 2. The survey was piloted in 56 members of the Australian public, recruited through an Internet-based survey panel (Dynata). Pilot results were used to update Bayesian priors and experimental design.

Fig. 2: Choice task example.
figure 2

Under which situation would you choose your child to have a genomic test? You can choose either ‘Situation 1’, ‘Situation 2’, or ‘I would not like my child to have a genomic test’.

Full size image

We recruited participants across Australia over the age of 18, stratified by age, gender, and income, through the market research company. The sample size was determined based on the S-efficiency measure, which provides the minimum sample size required for obtaining statistically significant estimates at the 95% confidence level when parameter priors are correctly specified [37]. As recommended [38], preferences and values of the general public were sought on the basis that they are the taxpayers and potential users of healthcare. To enhance respondents’ familiarity and understanding of the paediatric critical care context, participants were invited to read the real stories of two families who had a child admitted to intensive care with suspected genetic condition published in the Australian Genomics website. We also administered the survey to families with lived experience of rapid genomics recruited within the Australian Genomics Acute Care research programme [9]. Families with experience of a critically ill infant or child completed the same choice tasks but the background survey information was adjusted given their experience and familiarity with genetic conditions and rapid genomic testing.

All survey participants were randomised to one of the four blocks of the survey, namely to an identical version of the survey that included 12 out of the 48 choice tasks. Within each block, we also randomised the order in which the two genomic testing situations were presented to avoid a ‘left to right’ bias. A final randomisation was applied to a survey version with or without pictogram for the attribute ‘Number of children who receive genetic diagnosis’ to control for potential framing effect. The final version of the survey was approved by all members of the team, the focus group members, and a plain language advisor. The final survey questions can be found in the online appendices.

Choice data analysis

A panel error component mixed logit model was estimated using NLOGIT 6 (Econometric Software, Inc., Waverton, NSW, Australia). This model accommodates unobserved heterogeneity of preferences among respondents, instead of assuming that preferences are homogeneous within the group, and allows for a correlation between the two genomic testing situations described in choice tasks, since both indicate a preference towards genomic testing [27].

Heterogeneity of preferences among respondents is captured using random parameters with a specified probability distribution. Participant choices are used to allocate individual preferences within the distribution and derive individual-specific parameter estimates. A constrained triangular distribution was used for the cost attribute [27]. A normal distribution was used for the other parameters. Random parameters were estimated using 1000 standard Halton draws. All attributes were coded as continuous to enable estimations of incremental benefits within a cost–benefit analysis apart from the attribute ‘Time between test initiation and results’ where dummy coding was used to reflect non-linearities between utility and attribute levels. We determined the relative importance of each attribute using importance scores, which were calculated by comparing the range in estimated utility between best and worst attribute levels divided by the sum of the utility ranges across attributes.

We estimated marginal WTP values for each of the DCE attributes using the unconditional population moments estimates, which were further validated based on conditional (individual-level) estimates [27]. The incremental WTP for rapid and ultra-rapid genomic testing relative to standard care was then calculated using the compensating variation formula [39], based on published estimates of diagnostic yield, clinical utility, and turnaround times for results [6,7,8,9]. Estimates were based on population simulated data incorporating identified heterogeneity of preferences [27], using robust regressions. Robust regressions were used given then presence of outliers in the data. WTP values are reported in Australian and US dollars (using January 2, 2020 Reserve Bank of Australia exchange rate of 0.70).

To further understand preference heterogeneity among members of the Australian public, a latent class model was estimated [40]. Latent class models split the sample into a finite number of groups (classes). Each group has a different set of preferences but individuals within groups are assumed to have homogenous preferences. The model was estimated using four classes based on the Bayesian information criterion. A fractional logistic regression was then used to explore whether sociodemographic and attitudinal characteristics were significantly associated with class membership [41].

Results

The study included 522 members of the Australian public (cooperation rate = 75%) and 25 people with a lived experience of a critically ill child and rapid genomics (response rate = 25%). The mean age among the general public sample was 46 years (SD = 16). Of the 522 participants, 268 (51.3%) were female, 312 (59.8%) were parents and married or in a de facto relationship, and 195 (37.4%) had acquired a university-level education (Table S1). As shown in Table S1, 32.6% (n = 170) of the sample reported to have an experience, either personal or from a close family member or friend, of a genetic condition, and 22.6% (n = 118) reported an experience of genetic or genomic testing. About 40% (n = 204) of the sample had heard about genomic testing before receiving the survey. For the sample of participants with a lived experience of paediatric critical care and rapid genomics, mean age was 36 years (SD = 7), and 68% had received a molecular diagnosis for their child’s condition. Most participants in this sample (96%) were female, with 44% of participants having obtained a university-level education. Additional information about the two samples is provided in the online appendices (Table S1 and Fig. S1).

The members of the public demonstrated strong preferences for all DCE attributes (Table 2). Apart from the expected disutility associated with the cost attribute, as evidenced by the negative coefficient, there is significant utility gain associated with each of the remaining attributes. Participants had preference for more children being diagnosed and for higher chance of improving the process of child’s medical care and health outcomes. They also showed preference for faster result turnaround times. The standard deviation of all parameters is statistically significant, which indicates heterogeneity of preferences within the sample. Despite the small sample size of people with lived experience of a critically ill child, statistically significant preferences were identified across all attributes apart from the reduction in the time between test initiation and results from 6 months to 3 months (Table 3).

Table 2 Marginal utilities and willingness to pay (WTP)—preferences of the public.
Full size table
Table 3 Marginal utilities and willingness to pay (WTP)—preferences of people with lived experience of acute care genomics.
Full size table

The marginal WTP estimates for each attribute in the two samples are shown in Tables 2 and 3. On average, the public was willing to pay AU$48 (95% CI: AU$36–$57) per additional child in a hundred receiving genetic diagnosis, AU$95 (95% CI: AU$81–$105) for every percentage point increase in the chance of improving the process of the child’s medical care, and AU$354 (95% CI: AU$270–$436) for every percentage point increase in the chance of improving the child’s health outcomes (Table 2). Compared to the baseline turnaround time for results of 6 months, the public was, on average, willing to pay AU$612 (95% CI: AU$50–$1152) to receive the genomic test results in 12 weeks, AU$2951 (95% CI: AU$2404–$3432) to receive the results in 2 weeks, and AU$3527 (95% CI: AU$2943–$4080) to receive the results in 2 days. Similar values were observed among people with lived experience of a critically ill infant or child (Table 3), except for the attribute ‘Number of children who receive genetic diagnosis’ for which they were willing to pay on average AU$75 (95% CI: AU$36–$113), a value that is 1.5 times higher than the estimated value for the public. This difference is also evidenced in the importance scores of this attribute (Tables 2 and 3).

The estimated incremental value of rapid and ultra-rapid genomic testing in the critical paediatric care context is shown in Table 4. Based on published evidence for their diagnostic utility, clinical utility, and result turnaround times, as detailed in Table 4, we estimated an average WTP of AU$9510 (95% CI: AU$8684–$10,336) (US$6657 [95% CI: US$6079–$7235]) for rapid genomic testing, and AU$11,000 (95% CI: AU$10,078–$11,922) (US$7700 [95% CI: US$7055–$8345]) for ultra-rapid genomic testing relative to standard diagnostic care. The estimated WTP for rapid and ultra-rapid testing among people with experiences of critical paediatric care was on average AU$715 and AU$500 higher, respectively (Table 4).

Table 4 Uptake and willingness to pay (WTP) for rapid and ultra-rapid genomic testing (GT) relative to standard care (SC) for critically ill infants and children.
Full size table

Preference heterogeneity among the members of the Australian public can be more explicitly described based on the results of the latent class model, which indicates that the sample can be split into four homogenous classes (Table S2). Class 1, representing 40% of the sample, includes people who value all outcomes of genomic testing, and particularly, diagnostic and health outcomes. This class would be willing to pay on average AU$178 (US$125) per additional child in a hundred receiving genetic diagnosis, and AU$1887 (US$1321) for every percentage point increase in the chance of improving the child’s health outcomes. As shown in Table S3, being young, male, having positive attitude towards genomic testing, and having low familiarity with how genetic conditions impact on the lives of people who have them and those around them are characteristics associated with membership in class 1.

Class 2, representing 25% of the sample, includes people who value a timely diagnosis regardless of the likelihood of improving the child’s health outcomes (Table S2). This class would be willing to pay $9930 (US$6951) for a result turnaround time of 2 days. Being female, having heard about genomic testing, being familiar with the impact of genetic conditions, and having positive attitude towards genomics are characteristics associated with membership in class 2 (Table S3). Class 3, representing 25% of the sample, includes people who value diagnosis and improvement in the process of medical care, but are willing to pay less for these outcomes relative to the previous two classes. Being older, with no experience of genetic conditions, and less positive attitudes towards genomic testing are characteristics associated with membership in class 3. Class 4, representing 10% of the sample, includes people who have a preference against genomic testing and would not take it up in a critical care context. This is evidenced by the large negative genomic testing constant in Table S2 as well as the association between less positive attitudes towards genomic testing and class 4 membership (Table S3). In a real-world setting, this proportion has also been found to be 10% [42].

Discussion

This study quantified the utility of genomic testing in a critical paediatric care context from the perspective of the Australian society and the perspective of families with lived experiences of critically unwell children and rapid genomic testing. Both society and people with lived experiences had strong preferences for higher diagnostic yield and clinical utility, faster result turnaround times, and lower cost. Rapid reporting of results was found to be an important driver of utility. We estimated that society on average would be willing to pay an additional AU$9510 (US$6657) and AU$11,000 (US$7700) for rapid and ultra-rapid genomic testing relative to standard critical care. The corresponding WTP estimates among people with lived experiences were AU$10,225 (US$7158) and AU$11,500 (US$8050), respectively.

These findings are consistent with previous research on stated preferences for genomic testing in the context of paediatric genetic conditions. A study by Marshall et al. [24] elicited preferences for genomic testing among parents of children with genetic conditions. The study demonstrated that parents placed high value on the chance of diagnosis, the time required to obtain an answer, and the (positive or negative) implications of diagnosis, particularly the benefits related to improving knowledge and changing medical management or accessing condition-specific services [24]. The study concluded that parents on average would be willing to pay US$4943 for genomic testing relative to operative procedures. A study by Goranitis et al. [25] elicited preferences from the Australian public to estimate the WTP for genomic testing in complex paediatric neurological disorders of suspected genetic origin. Preferences were elicited for the following eight attributes: number of children who receive a genetic diagnosis; chance of improving the process of child’s medical care; availability of treatments; time until your child does the test; time between test and results; enabling access to other services and professional or peer support; enabling access to clinical trials; and cost of testing. The study identified strong preferences across all attributes and estimated that the public on average would be willing to pay US$3955 (95% CI: US$3850–$4060) for genomic testing relative to standard care [25]. Another study by Goranitis et al. [26] utilized a generic design to elicit preferences and values for genomic testing across childhood and adult-onset genetic conditions. For childhood conditions, the study estimated that the average societal WTP for genomic testing ranged between US$3830 and $10,675 depending on the risks and benefits involved. All these studies demonstrate the intrinsic value that parents and the public place on information, the timeliness of information, and potential clinical, non-clinical, and process outcomes that may accrue from the diagnosis. However, the higher estimates identified in this study essentially reflect the time-critical nature of the intensive care context. Indicatively, reducing the time to genomic results by 6 months is valued on average $US2470 in paediatric critical care, which is 5.3 times higher than the same time reduction in the context of complex neurology [25].

This is the first study to explore the value of genomic testing in critically ill infants and children and the importance that people place on rapid and ultra-rapid return of results. The study benefited from a large sample of adults from the general population in Australia who are the main stakeholders of the healthcare system [38], and a sample of people with lived experience of rapid genomic testing in their critically ill child. There are, however, limitations. First, in order to obtain an Australian-wide sample, participants were recruited through a professional survey organisation. Although this method may introduce self-selection or incentive biases, it is widely used and has been shown to be valid and reliable [43]. Second, there is evidence highlighting a discrepancy between descriptive and experience-based choices [44, 45], and the challenge of establishing external validity due to hypothetical bias and the lack of comparable revealed preference data. Nevertheless, the DCE survey was developed with support from community representatives with lived experience of genetic conditions, professionals specialising in clinical genomics, and a plain language advisor, and included the real stories of two families who had an experience of a critically ill child with a genetic condition. This process ensured high face validity despite the hypothetical nature of DCEs [46]. Third, the DCE was designed and analysed with the objective to generate WTP values for rapid and ultra-rapid genomic testing with the purpose of informing a cost–benefit analysis. To be able to capture marginal differences across tests and in a way that allows recalculation of values in light of new evidence for diagnostic and clinical utility of genomic testing, continuous coding was applied to all attributes apart from the attribute ‘Time between test initiation and results’. Continuous coding, however, does not capture a possibly non-linear relationship between attribute levels and utility. Finally, in severe genetic conditions a diagnosis may form part of a decision to redirect care towards palliation. Given the complex ethical issues associated with redirection of care towards palliation [47], this was not explicitly considered in the DCE. The ‘Time between test initiation and results’ attribute, however, does capture the disutility that people place on being in state of uncertainty.

The importance of considering health and non-health outcomes when informing funding and reimbursement decisions in healthcare is being increasingly recognised [48, 49]. This is particularly important in the context of critically ill children, where standard preference-based measures used in health economics are generally not applicable due to the very young ages of children involved and the large non-health benefits generated for parents. The findings of this work have important implications for the prioritisation and implementation of rapid genomic testing in paediatric critical care. Knowledge of the preferences of the public and people with lived experience of rapid genomic testing enables implementation that provides high value clinical care. This DCE provided a unique opportunity to further quantify the welfare gain from the implementation of genomic testing in critically ill children with genetic conditions. These findings can be directly compared with the economic consequences of implementing genomic testing through a cost–benefit analysis framework. Our findings demonstrated that rapid genomic testing is highly valued by both the Australian public and people with lived experiences of rapid genomic testing in their child. The evidence for the marginal utility of the different value components of genomic testing now enables decision-makers to optimise the implementation of genomic testing, balancing the trade-offs between reducing the time to test results and cost.