Introduction

Patient-reported outcomes (PROs) are widely used as outcome measures in clinical trials of pain treatments. Indeed, given that pain can only be measured subjectively, studies of pain are entirely reliant on self-reporting [1]. The assessment of pain is incorporated in different ways in PROs, ranging from generic measures of health-related quality of life, such as EQ-5D [2] in which pain/discomfort is only one of five dimensions measured, to tools that are specific to the assessment of pain, such as the Brief Pain Inventory [3]. One important consideration when using such measures in multi-country studies is that factors such as race, ethnicity, language, and culture can potentially affect responses to PRO instruments [4–6].

With regard to pain reporting, there is evidence that this can vary quite widely across countries; for example, the results of a survey published in 2014 showed that pain reporting and treatment rates were lower in China (6.2% and 28.3%, respectively) and Japan (4.4% and 26.3%, respectively) than in the other countries involved (≥14.3% and 35.8%, respectively) [7]. Substantial variations between countries in the rate of pain reporting have also been reported in primary care [8] and among patients with cancer [9]. Within Europe, studies have also shown variation in pain reporting; using the pain/discomfort dimension of the EQ-5D, rates of respondents declaring no pain/discomfort varied from 65% in France to 79.5% in Spain [10]. At least some of the difference between countries may depend on cultural differences in pain response rather than on differences in objective levels of pain. For example, in experimental studies, Japanese subjects provided lower pain ratings for equivalent ‘objective’ levels of pain than European subjects [11], whereas other studies have shown that Euro-Americans consider seeking pain relief more acceptable than Japanese respondents [12]. However, evidence from Uki et al. suggests that Japanese patients with cancer reported high levels of pain with inadequate pain management [13].

The EQ-5D is one of the most widely used generic preference-based measures of health status. It has been translated into numerous languages and is available in an increasing number of country-specific utility-based value sets. Nevertheless, little attention has been paid to whether the effects described earlier in relation to pain affect the self-rating and valuation of pain/discomfort on EQ-5D and whether such data can be compared and aggregated across countries. In one of the few such analyses performed, Tsuchiya et al. compared the results of the UK and Japanese EQ-5D-3L valuation data and noted that the two datasets were positively correlated [14]. Furthermore, Japanese time trade-off (TTO) values were consistently higher than those from the UK, except for mild states.

The availability of self-reported and valuation EQ-5D-3L data from Japan and the UK together with self-reported and valuation data from England and Japan for the latest version of the instrument, the EQ-5D-5L, makes it possible to explore these questions in more depth. Self-reported data were also available for the EQ-5D-5L from Spain. The primary aim of this study was to investigate whether available EQ-5D data provide evidence of systematic differences in the way respondents in Japan self-report and value health using EQ-5D compared with respondents from England/UK and Spain, with a particular focus on the pain/discomfort dimension.

Methods

The EQ-5D

The EQ-5D consists of a descriptive system and a visual analog scale (EQ-VAS). Respondents rate their health on the EQ-5D descriptive system and assess their overall health on the EQ-VAS.

There are two versions of the instrument for use in the adult population, EQ-5D-3L and EQ-5D-5L. Both measure health using a descriptive system with five dimensions (mobility, self-care, usual activities, pain/discomfort, and anxiety/depression). The original version of EQ-5D (EQ-5D-3L) uses three levels of severity (no, some, extreme problems or unable to) in each dimension. To increase the instrument’s sensitivity, a new version of the instrument (EQ-5D-5L) was developed with five levels of severity (no, slight, moderate, severe, extreme problems or unable to) [15]. Self-ratings on the descriptive system are summarized as a five-number ‘code’ where each number reflects the severity level on the individual dimensions. Each ‘code’ represents a unique health state, with the state 11111 representing full health. There are 243 EQ-5D-3L health states (243 = 35) and 3125 EQ-5D-5L health states (3125 = 55). By selecting one level of severity in each dimension on either version of the EQ-5D, respondents assign themselves one out of all possible health states as a description (known as a ‘profile’) of their own health.

The EQ-VAS consists of a vertical scale with anchor points of 0 (worst possible health) and 100 (best possible health). The respondent marks a point on the scale to show how they perceive their overall health.

In addition to providing these two types of self-reported data, the EQ-5D is frequently used in conjunction with health state valuation techniques such as TTO or discrete choice experiments to generate preference-based societal weights for each of the individual health states generated by the descriptive system [16]. It is recommended that weights be obtained for individual countries, as values assigned to health states might differ between countries for cultural or other reasons [17].

Data

Valuation and self-reported health data obtained from the EQ-5D-3L valuation surveys in the UK and Japan as well as EQ-5D-5L valuation surveys in England and Japan were used [14, 18–20]. At the time the present study was carried out, self-reported health data on EQ-5D-5L were also available for Spain [21]. In all cases, the samples included were intended to be representative of the general population of the country.

Socio-demographic and health-related characteristics available in each of the five datasets included age, gender, and respondents’ experience of serious illness (in themselves, a family member, or others). The same background characteristics were recorded in each of the five datasets.

The Japanese EQ-5D-3L valuation study was a quasi-replication of the UK valuation study [14]. Each respondent valued the same set of the 17 health states, which were a subset of the 42 health states in the UK study [18]. The EQ-5D-5L valuation data used in the present analysis were collected during valuation studies carried out in Japan, England, and Spain using the EuroQol valuation technology (EQ-VT) software, which was developed specifically for the EQ-5D-5L value set studies [16]. An identical methodology based on computer-assisted personal interviews (CAPI) was used in all countries. Each respondent was asked to provide TTO values for a block of 10 health states out of the 86 health states selected for direct valuation. All interviewers received training on administration of the CAPI and EQ-VT. Only the observed TTO values (rather than the value sets modeled from those data) were used in this study.

A summary of the characteristics of each survey is provided in Table 1.

Table 1 Characteristics of the five survey datasets
Full size table

Statistical analysis

There were five parts to the statistical analysis. First, respondents’ socio-demographic and health-related characteristics were compared across the five datasets. Second, self-reported data on the descriptive systems of EQ-5D-3L and EQ-5D-5L were compared to investigate whether there were systematic differences between countries that could not be explained by differences in sample characteristics on age or gender, particularly in regard to reporting of pain/discomfort. Third, the impact of the five dimensions of the descriptive system on EQ-VAS scores was analyzed to determine, in particular, the contribution of the pain/discomfort dimension to VAS scoring. Fourth, valuation data were used to explore whether respondents in Japan and the UK for the EQ-5D-3L valuation studies and respondents in Japan and England for the EQ-5D-5L valuation studies have different stated preferences in terms of their willingness to trade off time in TTO tasks, particularly in relation to health states involving pain/discomfort. Fifth, the linked self-reported and valuation datasets for the EQ-5D-3L and EQ-5D-5L valuation studies in the UK/England and Japan were analyzed to investigate whether there was any relationship between respondents’ self-reported pain/discomfort and the TTO values they assigned to the hypothetical health states.

The socio-demographic and health-related characteristics of the different samples were compared using t tests for age, and chi-squared tests for gender and the proportions of respondents who reported having experienced serious illness in themselves, a family member, or others.

Respondents’ self-reported EQ-5D data were analyzed by comparing the distribution of EQ-5D profile data by country and instrument version. The ceiling effect (measured by the proportion of respondents reporting the best possible health for EQ-5D), the number of EQ-5D profiles used, and the distribution of responses by dimension were also calculated and compared across countries and EQ-5D versions. Adjustments for age and gender were made when comparing distributions on the descriptive system across countries. Using England as an example, to adjust for age difference, five age ranges were used (≤30, 31–45, 46–60, 61–75, and >75 years). For each age range, the proportion of respondents in England who reported full health was calculated followed by the weighted average of the five proportions. The weight for each age band is the proportion of respondents in that age band in Spain or Japan. Adjustments for any differences in gender distributions were made in the same way.

The extent to which responses on the five dimensions of the descriptive system explained self-reported overall health on the EQ-VAS was examined using ordinary least squares (OLS) regression methods and the results were compared between countries for both the 3L and 5L. Data on the five dimensions of the EQ-5D were recorded as continuous variables (1 for level 1, 2 for level 2, 3 for level 3, 4 for level 4, and 5 for level 5). To show the model’s goodness of fit, adjusted R-squared and results from residual analysis were reported after each regression.

To explore whether there were differences in TTO values in the valuation studies between respondents, in particular for pain/discomfort, EQ-5D-3L and EQ-5D-5L valuation data from the UK/England and Japan were analyzed. Respondents in the UK and Japan in the EQ-5D-3L studies yielded observed values for 42 health states in the UK and 17 health states in Japan. The EQ-5D-5L studies in Japan and England yielded observed values for 86 health states. TTO values in each version of the EQ-5D for those health states were compared between the two countries using t tests.

Among the 17 hypothetical health states in the EQ-5D-3L valuation data, there were three pairs of health states that only differed on the pain/discomfort dimension. Among the 86 hypothetical health states in the EQ-5D-5L valuation data, there were seven pairs of health states that only differed on the pain/discomfort dimension. For each version of the EQ-5D valuation studies, the difference in mean TTO values was compared between respondents by country (for those pairs that only differed in the pain/discomfort dimension) to gain insight into differences in how respondents in Japan and the UK/England value EQ-5D health states with respect to the pain/discomfort dimension.

Finally, respondents’ self-reported and valuation data were linked using respondents’ ID for the EQ-5D-3L and EQ-5D-5L data from Japan and the UK/England. Using both the self-reported and valuation data, the effect of self-reported pain/discomfort in explaining the TTO values was explored using the OLS regression analyses to model the TTO values for the five dimensions of the EQ-5D profile and self-reported pain/discomfort. All analyses were performed using STATA/MP 13.

Results

There was no statistically significant difference in mean age between respondents in the UK and Japan in the EQ-5D-3L data (P = 0.753), although the UK sample had a higher proportion of females and respondents who had experienced serious illness in themselves or in a family member, and a lower proportion of respondents who had taken care of others with a serious illness (P < 0.05). There was no statistically significant difference in mean age between respondents in Japan and Spain in the EQ-5D-5L datasets (P = 0.119); however, respondents in England were older than respondents in Japan and Spain (P < 0.05). The English sample also reported the highest proportion of females and respondents who had experienced serious illness themselves, in a family member, or who had taken care of others with a serious illness (P < 0.05) (Table 2).

Table 2 Socio-demographic and health-related characteristics of respondents in Japan and the UK for the EQ-5D-3L; Japan, England, and Spain for the EQ-5D-5L
Full size table

The most frequently self-reported profile using both versions of the EQ-5D was full health. The proportion of respondents reporting full health was highest in Japan for both versions of the EQ-5D, and the differences between other countries were statistically significant (P < 0.05). After adjusting for differences in age and gender in the EQ-5D-5L samples, the proportion of those reporting full health was still highest among respondents in Japan (66.5%), followed by Spain (54.9%) and England (53.8%). The reduction in the ceiling effect using the EQ-5D-5L compared with the EQ-5D-3L was similar in the UK/England (from 56.9% to 47.6% for the EQ-5D-3L and EQ-5D-5L, respectively) and Japan (from 77.2% to 66.5%).

Analysis of self-reported EQ-5D profile data showed that Japanese respondents employed a much smaller number of health profiles than respondents in other countries. Among the EQ-5D-3L datasets, three health states accounted for 90.1% of Japanese respondents compared with 12 health states in the UK (90.6%). The difference is even more marked for the EQ-5D-5L, where only four health states accounted for 91.4% of Japanese respondents compared with 80 health states in England (90.1%) and 16 health states in Spain (90.0%). Full EQ-5D profile distributions by country and EQ-5D version are available upon request from the authors.

Figure 1 shows the proportion of respondents reporting level one for the EQ-5D-3L by dimension and country. In the pain/discomfort dimension, 81.6% of respondents in Japan self-reported level one, compared with 67.0% of respondents in the UK. Figure 2 shows that more respondents in Japan self-reported level one for the EQ-5D-5L than in England and Spain in four dimensions. In the pain/discomfort dimension, a higher proportion of respondents in Spain (79.2%) self-reported level one than in Japan (73.2%) and England (58.4%), although after adjusting for differences in age and gender between the samples, the proportion of respondents reporting level 1 on the pain/discomfort dimension in England was approximately 64%.

Fig. 1
figure 1

The proportion of respondents in Japan and the UK who reported level one (no problems) by EQ-5D-3L dimension. MO mobility, SC self-care, UA usual activities, PD pain/discomfort, AD anxiety/depression

Full size image
Fig. 2
figure 2

The proportion of respondents in Japan, England, and Spain reporting level one (no problems) by EQ-5D-5L dimension. MO mobility, SC self-care, UA usual activities, PD pain/discomfort, AD anxiety/depression. Data on the pain/discomfort dimension for England include those adjusted by age distribution of Japan (Japan age) and Spain (Spain age)

Full size image

The results of modeling respondents’ EQ-VAS scores as a function of their self-reported EQ-5D profiles are shown in Table 3. EQ-VAS scores decreased when the severity of problems increased in any of the five dimensions, and this finding was consistent across countries and EQ-5D versions. Using EQ-5D-3L, pain/discomfort was the most important dimension in explaining respondents’ self-reported EQ-VAS scores in Japan (on average, a one level increase in the pain/discomfort dimension led to a decrease of 11.03 points on the EQ-VAS), whereas in the UK, it was the dimension of usual activities. Using EQ-5D-5L data, anxiety/depression was the most important dimension in explaining differences in EQ-VAS scores in England and Spain; however, in Japan, the most important dimension was usual activities. The adjusted R-squared and results from residual analyses for each model are reported in Table 3. The results from residual analyses suggest no evidence of multicollinearity in the five models. However, there is evidence of residuals with non-normal distributions, heteroscedasticity, and non-linear functional form for some specifications.

Table 3 Modeling self-reported EQ-VAS scores by country and EQ-5D version
Full size table

Table 4 reports the mean EQ-5D-3L TTO values for the 17 hypothetical health states valued by respondents in the UK and Japan. In Japan, none of the mean TTO values for the 17 health states were below zero (i.e., none of them were considered as being worse than dead). By contrast, six of the 17 health states in the UK had negative mean TTO values. TTO values for the five mildest health states (11112, 11121, 11211, 12111, 21111) were lower in Japan than in the UK, with the differences being statistically significant at the 5% significance level. However, Japanese TTO values for the remaining 12 more severe health states were all higher than UK values for those health states. The differences were statistically significant at the 5% significance level for 11 health states, but not for health state 22222.

Table 4 Comparing the mean TTO values for the 17 hypothetical EQ-5D-3L health states between respondents in Japan and the UK
Full size table

Table 5 reports the mean EQ-5D-5L TTO values for the 86 hypothetical health states valued by respondents in Japan and England. For respondents in both countries, only the worst state 55555 was assigned a mean TTO value below zero. Mean TTO values were higher in Japanese respondents for 63 of the 86 hypothetical health states, with 19 of those differences being statistically significant (P < 0.05). No clear pattern was observed between the severity of health states and the presence of higher values from Japanese respondents; the 19 health states included mild states (e.g., 11112) and severe states (e.g., 55555). In only one case was a statistically significant higher value observed for English raters (state 35332).

Table 5 Comparing the mean TTO values for the 86 hypothetical EQ-5D-5L health states between respondents in Japan and England
Full size table

Three pairs of health states in the EQ-5D-3L valuation data in the UK and Japan differed only on the pain/discomfort dimension. Analysis of those states showed that Japanese respondents traded off less time to avoid problems in the pain/discomfort dimension than UK respondents. The biggest between-country difference in mean TTO values was reported between health states 11121 and 11131, which would represent a decrease of 0.65 in TTO values in the UK compared with 0.15 in Japan.

Seven pairs of health states in the EQ-5D-5L in the English and Japanese valuation studies differed only in the pain/discomfort dimension. Specifically, four pairs differed between level 1 (no problem) and level 2 (mild problem), two pairs differed between level 1 (no problem) and level 4 (severe problem), while one pair differed between level 3 (moderate problem) and level 4 (severe problem). Comparing the mean TTO values between respondents in the two countries showed that Japanese respondents traded off either similar or less time to avoid problems in pain/discomfort than English respondents. The biggest difference in mean TTO values was reported between health states 12334 and 12344, which would represent a decrease of 0.19 in TTO values in England compared with 0.10 in Japan.

Regression analysis of the linked self-reported and valuation datasets showed that respondents’ self-reported pain/discomfort was not significant in explaining the TTO values in Japan for EQ-5D-3L (P = 0.395) and EQ-5D-5L (P = 0.299), nor the UK for EQ-5D-3L (P = 0.159). However, it has significant positive effect in explaining the EQ-5D-5L TTO values in England (P < 0.05).

Discussion

This is the first study to carry out an in-depth examination of the comparability of EQ-5D self-rated health status and valuation data from Japanese and European respondents, with a particular focus on pain/discomfort. A number of findings were clear from the empirical analyses in this study.

First, respondents in Japan tend to report better health in general than respondents in England/UK and Spain. Second, with respect to pain/discomfort, respondents in Japan reported problems less frequently than respondents in England/UK, but slightly more frequently than respondents in Spain. Third, Japanese respondents used a much smaller number of health states to describe their health than respondents in either of the other two countries, and Spanish respondents also used substantially fewer health states than respondents in England. Fourth, in the EQ-5D-3L valuation study, respondents in Japan were more willing to trade off time for the mildest health states, but less willing to trade off time for the severe health states compared with respondents in the UK. For nearly three-fourths of the EQ-5D-5L health states for which values were obtained, Japanese respondents’ values were higher than those from English respondents. However, in contrast with EQ-5D-3L values, there was no clear pattern between this and the severity of the states. Fifth, in the EQ-5D-3L and EQ-5D-5L valuation studies, Japanese respondents were willing to trade off less time to avoid problems in the pain/discomfort dimension than respondents in England/UK. However, the differences in TTO values between respondents in Japan and England are much smaller than in the EQ-5D-3L valuation study.

It is not clear where these differences stem from, though similar findings have been reported previously. For example, in a comparison of EQ-5D results from 20 countries in a diabetes clinical trial, researchers found substantial variation in the reporting of functional health problems, but noted that the variation could not be explained by differences in demographic variables, clinical risk factors, or rates of complications [22]. They suggested that the unexplained variability meant there were important problems of comparability across settings.

One possible cause of the differences found here is that the way terms used to describe health, e.g., the severity labels, varies across countries. For example, Luo et al. found that the interpretation and use of EQ-5D-5L response labels (e.g., ‘slight’, ‘moderate’, and ‘severe’) varied across Chinese, Malay, and English speakers in Singapore [23], whereas the English version gave similar outcomes in Chinese and non-Chinese English speakers in the same country [24], suggesting that there was no effect of culture on responses. Although a strict protocol is followed in producing other language versions of EQ-5D [25], it may not always be possible to find identical terms in all languages. There is also evidence suggesting that Japanese respondents might be less willing to report pain than those in Europe, possibly due to a tendency within the Japanese culture for pain to be repressed and controlled rather than shared or expressed [26].

Our findings on the reporting of pain/discomfort coincide with those of earlier multi-country studies that showed a tendency towards lower rates or intensity of self-reported pain in Japan than in other countries [7, 8, 27]. Despite lower rates of self-reported pain/discomfort in Japan, we found that this was the most important dimension in explaining respondents’ self-reported EQ-5D-3L VAS scores. There are two possible explanations for this discrepancy. First, while the EQ-VAS scores and self-reported EQ-5D profiles measure how good or bad respondents rate their currently experienced health status, the TTO valuation task evaluates health states that are hypothetical to the respondents. Second, the tasks involved EQ-VAS scores rating and TTO valuation, which are individually very different. It is possible for a respondent to rate a health state as poor on the EQ-VAS, but still not be willing to trade off any life years to avoid it (e.g., because of religious beliefs about the sanctity of life, being the primary caregiver to a small child, or having a very low personal discount rate). As the EQ-5D profile variables are treated as continuous, the importance of each EQ-5D dimension in explaining the EQ-VAS reflects the average effect in a dimension between two neighboring levels. An alternative approach is to treat the EQ-5D profile variables as dummies (i.e., one for each level and dimension). However, this would leave some categories with rather small sample sizes, particularly for severe levels (n < 5). Those results from the EQ-5D-3L data were not confirmed using EQ-5D-5L data. Given the design of the current study, it was not possible to determine whether the difference in findings was due to changes in methods or perceptions of the importance of pain/discomfort over time. It should also be noted that the lowest rates of pain/discomfort were observed in Spain. Other studies have also reported relatively low rates of self-reported problems on the EQ-5D descriptive system in the general population in Spain compared with other European countries although not on the EQ-VAS [10]. Similar findings have been reported for Spain using other instruments, such as the Brief Pain Inventory [28].

The comparison of valuation data also showed differences between countries. The Japanese EQ-5D-3L valuation data showed a tendency to compress towards the middle of the scale. A mid-range response style and lower levels of extreme response style have been reported in some studies in Japanese subjects [29–31], though it is not clear whether such an effect may also be present in valuation studies. Furthermore, we found that respondents in Japan were less willing to trade off time to avoid pain/discomfort on the EQ-5D-3L than respondents in the UK. It should be noted that the 17 health states valued by the Japanese respondents were a subset of the health states in the UK valuation study. The TTO values may be influenced by the mix of severity in the set of states presented. This may affect observed differences in TTO values.

This compression of values in Japan relative to UK values was no longer observed when analyzing results from the EQ-5D-5L. Only the worst state (i.e., 55555) was rated worse than death in both countries. Almost three-quarters of the EQ-5D-5L health states were given higher values by Japanese respondents compared with English respondents, indicating more reluctance to trade off time among Japanese respondents. However, unlike with the EQ-5D-3L data, these higher ratings were spread across all levels of severity. It is possible that these differences are due to changes in the methods that were used in the TTO valuation tasks between the two versions of EQ-5D, or changes in perceptions of health states, and/or relative importance assigned to different dimensions over time. However, it is not possible to answer it definitively here.

Limitations

Ideally, samples from Japan and European countries used to explore differences in the self-reporting and valuation of pain would have identical distributions for all factors that might influence results. However, it was not possible to control for all relevant variables, although we controlled for the effects of age and gender. It remains unclear whether differences in the rates of health problems between countries and other unobserved characteristics may have led to the differences we observed.

Furthermore, the EQ-5D-3L data used in this study were collected in the 1990s and may no longer be applicable to the present populations in the UK and Japan; however, we do not consider this to be a limitation of the present analysis as we were interested in comparing results between countries and not in exploring whether the data collected then would be relevant today. The fact that it was possible to compare findings from two different variants of the instrument at two different time points could in fact be considered a strength of the study because it gives an indication of the robustness of the results.

The possible misspecifications for modeling EQ-VAS scores by the EQ-5D dimensions should be noted. Although the assumption of normality does not hold in the five models and, as a consequence, will have an impact on the P values, the estimated coefficients themselves will still be consistent. Four models reported heteroscedasticity in the residuals and two models reported non-linear functional forms. These misspecifications might be explained by variables that are not included in modeling the EQ-VAS, but have an impact on the EQ-VAS, such as other health dimension(s) that are not covered by EQ-5D. If those variables are correlated with the EQ-5D dimensions, our estimated coefficients could be biased. Similar issues have been observed in previous studies [18, 32, 33].

Finally, only data from two European countries were available for the present study and it is not clear whether results can be extrapolated to respondents in other Western countries.

Implications

Our findings have a number of implications. First, care should be taken when comparing and aggregating clinical data on pain between different countries, because respondents may use different criteria when responding, which could potentially lead to the same treatment being more or less effective in different countries. Second, the differences between respondents in Japan and European countries in self-reported and valuation behaviors could have a substantial effect on the results of cost–utility analyses. For instance, while applying the EQ-5D-3L instrument, the compression of values on the utility scale and the better baseline pain scores observed in Japan may result in relatively small improvements with treatment. Third, what constitutes a minimally important difference for EQ-5D index may be different between Japan and other countries. Fourth, if the findings related to pain/discomfort also applied to other pain measures used as inclusion criteria for clinical trials, then they might lead to questions about whether identical inclusion criteria for clinical trials are in fact being used across countries.

Conclusions

This study provides prima facie evidence of differences between Japan, UK/England, and Spain in the self-reporting and valuation of health, including pain/discomfort, when using EQ-5D in general population samples. The findings suggest the need for caution when comparing and/or aggregating EQ-5D data across the countries. Specifically designed studies, including the use of qualitative research and vignette techniques [34], would be helpful in exploring these issues further and confirming the findings.