Next Article in Journal
Models for COVID-19 Daily Confirmed Cases in Different Countries
Next Article in Special Issue
Study of Reversible Platelet Aggregation Model by Nonlinear Dynamics
Previous Article in Journal
Buckling of Tapered Heavy Columns with Constant Volume
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Global Hypothesis Test to Compare the Predictive Values of Diagnostic Tests Subject to a Case-Control Design

1
Department of Epidemiology and Public Health Research Unit and URMCD, University of Nouakchott Alaasriya, Nouakchott BP 880, Mauritania
2
Department of Statistics (Biostatistics), School of Medicine, University of Granada, 18016 Granada, Spain
*
Author to whom correspondence should be addressed.
Mathematics 2021, 9(6), 658; https://doi.org/10.3390/math9060658
Submission received: 17 February 2021 / Revised: 14 March 2021 / Accepted: 17 March 2021 / Published: 19 March 2021
(This article belongs to the Special Issue Mathematics in Biomedicine)

Abstract

:
Use of a case-control design to compare the accuracy of two binary diagnostic tests is frequent in clinical practice. This design consists of applying the two diagnostic tests to all of the individuals in a sample of those who have the disease and in another sample of those who do not have the disease. This manuscript studies the comparison of the predictive values of two diagnostic tests subject to a case-control design. A global hypothesis test, based on the chi-square distribution, is proposed to compare the predictive values simultaneously, as well as other alternative methods. The hypothesis tests studied require knowing the prevalence of the disease. Simulation experiments were carried out to study the type I errors and the powers of the hypothesis tests proposed, as well as to study the effect of a misspecification of the prevalence on the asymptotic behavior of the hypothesis tests and on the estimators of the predictive values. The proposed global hypothesis test was extended to the situation in which there are more than two diagnostic tests. The results have been applied to the diagnosis of coronary disease.

1. Introduction

The main parameters to assess and compare the accuracy of binary diagnostic tests (BDTs) are sensitivity and specificity. The sensitivity (Se) is the probability of the result of the BDT being positive when the individual has the disease, and the specificity (Sp) is the probability of the result of the BDT being negative when the individual does not have the disease. Other parameters that are used to assess and compare two BDTs are the predictive values (PVs). The positive predictive value (PPV) is the probability of an individual having the disease when the result of the BDT is positive, and the negative predictive value (NPV) is the probability of an individual not having the disease when the result of the BDT is negative. The PVs represent the accuracy of the diagnostic test when it is applied to a cohort of individuals, and they are measures of the clinical accuracy of the BDT. The PVs depend on Se, Sp and on the disease prevalence (p), and are easily calculated applying Bayes Theorem, i.e.,
P P V = p × S e p × S e + 1 p × 1 S p   and   N P V = 1 p × S p p × 1 S e + 1 p × S p
whereas the Se and the Sp quantify how well the BDT reflects the true disease status (present or absent), the PVs quantify the clinical value of the BDT, since both the individual and the clinician are more interested in knowing how probable it is to have the disease given a BDT result.
The comparison of the performance of two binary diagnostic tests is a topic of special importance in the study of statistical methods for the diagnosis of diseases. This comparison is made through a paired-design or through a case-control design. The paired design consists of applying the two BDTs and the gold standard to all of the individuals in a single sample. The case-control design consists of applying the two BDTs to all of the individuals in two samples, one made up of individuals who have the disease (case sample) and another made up of individuals who do not have the disease (control sample). The advantages and disadvantages of the case-control design over the paired design can be seen in the book by Pepe [1]. Summarizing, the case-control design has some advantages over the paired design: (a) the case-control design is more efficient in terms of sample size requirements, (b) case-control studies allow for the exploration of subject-related characteristics of the test. Nevertheless, the case-control design has the disadvantage is that by using it we cannot estimate the prevalence of the disease.
In paired designs, the comparison of PVs has been the subject of several studies. Bennett [2,3], Leisenring et al. [4], Wang et al. [5] and Kosinski [6] studied hypothesis tests to independently compare the PPVs and the NPVs of two BDTs. Moskowitz and Pepe [7] studied the estimation of the PVs through a confidence region. Roldán-Nofuentes et al. [8] studied the joint comparison of the PPVs and NPVs of two BDTs, and proposed a global hypothesis test based on the chi-square distribution to simultaneously compare the PVs of two BDTs.
In a case-control design, Mercaldo et al. [9] have studied the estimation of the PVs of a BDT, assuming that the prevalence of the disease (p) is known. The prevalence can be known from other studies, such as population studies of health services, cohort studies, etc. Mercado et al. have verified through simulation experiments that the confidence interval with the best asymptotic behavior is the logit interval, whose equations are:
P P V exp logit P P V ^ z 1 α / 2 V ^ a r logit P P V ^ 1 + exp logit P P V ^ z 1 α / 2 V ^ a r logit P P V ^   ; exp logit P P V ^ + z 1 α / 2 V ^ a r logit P P V ^ 1 + exp logit P P V ^ + z 1 α / 2 V ^ a r logit P P V ^
and
N P V exp logit N P V ^ z 1 α / 2 V ^ a r logit N P V ^ 1 + exp logit N P V ^ z 1 α / 2 V ^ a r logit N P V ^   ; exp logit N P V ^ + z 1 α / 2 V ^ a r logit N P V ^ 1 + exp logit N P V ^ + z 1 α / 2 V ^ a r logit N P V ^
where P P V ^ and N P V ^ are the estimators of the PVs calculated from Equation (1), z 1 α / 2 is the 100 1 α / 2 th percentile of the normal standard distribution, and the variances are:
V ^ a r logit P P V ^ = 1 S ^ e n 1 S ^ e + S ^ p n 2 1 S ^ p   and   V ^ a r logit N P V ^ = S ^ e n 1 1 S ^ e + 1 S ^ p n 2 S ^ p ,
where S ^ e and S ^ p are the estimators of sensitivity and specificity, n 1 is the size of the case sample and n 2 is the size of the control sample.
In this article, we extended the study of Mercaldo et al. [9] to the case of two BDTs, studying different hypothesis tests to compare the PVs of the two BDTs subject to a case-control design. Subject to a case-control design, the two BDTs are applied to all of the individual in two samples, one of n 1 individuals who have the disease (case sample) and another with n 2 individuals who do not have the disease (control sample). In this design, the sample sizes n 1 and n 2 are set by the researcher. The sample of individuals that have the disease is extracted from a population of individuals that have the disease (e.g., registers of diseases), and the control sample is extracted from a population of individuals who are known not to have the disease. As the PVs depend on the disease prevalence and subject to a case-control design the quotient n 1 / n 1 + n 2 is not an estimator of the prevalence, in order to estimate and compare the PVs subject to this design it is necessary to know the value of the prevalence of the disease. This value can be obtained from health surveys or from previous studies. Consequently, the methods of comparison of the PVs subject to a paired design cannot be applied when there is a case-control design. In Section 2, we study hypothesis tests to simultaneously compare the PVs of two BDTs subject to a case-control design. A global hypothesis test is studied to simultaneously compare the PVs of the two BDTs, i.e.,
H 0 : P P V 1 = P P V 2   and   N P V 1 = N P V 2 ,
and simultaneous comparison is also studied from individual hypothesis tests, i.e.,
H 0 : P P V 1 = P P V 2   and   H 0 : N P V 1 = N P V 2
each of them to the α error and also applying multiple comparison methods. In Section 3, simulation experiments are carried out to study the type I errors and the powers of the hypothesis tests proposed in Section 2, and we study the effect of the misspecification of the prevalence on the asymptotic behavior of the hypothesis tests proposed in Section 2 and on the estimators of the PVs. In Section 4, the results are applied to a real example on the diagnosis of coronary heart disease. In Section 5, the model proposed in Section 2 was extended to the situation in which we compare the PVs of more than two BDTs, and in Section 6 the results are discussed.

2. Global Hypothesis Test

Let us consider two BDTs, Test 1 and Test 2, which are applied to all of the individuals in two samples, one of n 1 individuals who have the disease (case sample) and another of n 2 individuals who do not have it (control sample). Let T 1 and T 2 be two binary variables that model the results of each BDT, in such a way that T i = 1 when the result of the corresponding BDT is positive and T i = 0 when it is negative. In Table 1, we can see the probabilities associated to the application of both BDTs to both types of individuals (cases and controls), as well as the frequencies observed.
Using the conditional dependence model of Vacek [10], the probabilities given in the table are written as:
ξ 1 j k = S e 1 j 1 S e 1 1 j S e 2 k 1 S e 2 1 k + δ j k ε 1 ,
and
ξ 2 j k = S p 1 1 j 1 S p 1 j S p 2 1 k 1 S p 2 k + δ j k ε 2 ,
with j , k = 0 , 1 . The parameter ε 1 ( ε 2 ) is the covariance between the two BDTs in cases (controls), where δ j k = 1 if j = k and δ j k = 1 if j k , and it is verified that 0 ε 1 Min S e 1 1 S e 2 , S e 2 1 S e 1 and 0 ε 2 Min S p 1 1 S p 2 , S p 2 1 S p 1 . If ε i = 0 then the two BDTs are conditionally independent on the disease status. In practice, the assumption of the conditional independence is not realistic, and therefore ε 1 > 0 and/or ε 2 > 0 . In terms of the probabilities ξ i j k , the sensitivities are written as:
S e 1 = ξ 111 + ξ 110   and   S e 2 = ξ 111 + ξ 101 ,
and the specificities are written as:
S p 1 = ξ 201 + ξ 200   and   S p 2 = ξ 210 + ξ 200 .
The estimators of sensitivities are S ^ e 1 = n 11 · / n 1 and S ^ e 2 = n 1 · 1 / n 1 , and the estimators of specificities are S ^ p 1 = n 20 · / n 2 and S ^ p 2 = n 2 · 0 / n 1 . The estimators of their variances are V ^ a r S ^ e 1 = S ^ e 1 1 S ^ e 1 / n 1 , V ^ a r S ^ e 2 = S ^ e 2 1 S ^ e 2 / n 1 , V ^ a r S ^ p 1 = S ^ p 1 1 S ^ p 1 / n 2 and V ^ a r S ^ p 2 = S ^ p 2 1 S ^ p 2 / n 2 . Therefore, the sensitivities and the specificities are estimated as proportions of marginal totals. In this way, in the case sample we are interested in the marginal frequencies n 11 · and n 1 · 1 , and therefore these frequencies are the product of a type I bivariate binomial distribution [11]. In an analogous way, from the control sample, the marginal frequencies n 20 · and n 2 · 0 are the product of a type I bivariate binomial distribution. In the individuals with the disease, the type I bivariate binomial distribution is characterized [11] by S e 1 , S e 2 and the correlation coefficient ( ρ 1 ) between T 1 and T 2 . In an analogous way, in the individuals who do not have the disease, the type I bivariate binomial distribution is characterized by S p 1 , S p 2 and the correlation coefficient ( ρ 2 ) between T 1 and T 2 . Therefore, the proposed model is a parametric model based on the distribution of the marginal frequencies in each 2 × 2 table. In the individuals with the disease (cases), the correlation coefficient between the two BDTs is:
ρ 1 = ξ 111 S e 1 S e 2 S e 1 1 S e 1 S e 2 1 S e 2 = ε 1 S e 1 1 S e 1 S e 2 1 S e 2 ,
and in the individuals who do not have the disease (controls), the correlation coefficient between the two BDTs is:
ρ 2 = ξ 200 S p 1 S p 2 S p 1 1 S p 1 S p 2 1 S p 2 = ε 2 S p 1 1 S p 1 S p 2 1 S p 2 .
It is easy to show that ε ^ 1 = n 1 n 111 n 11 · n 1 · 1 / n 1 2 , ε ^ 2 = n 2 n 200 n 20 · n 2 · 0 / n 2 2 , C ^ o v S ^ e 1 , S ^ e 2 = ε ^ 1 / n 1 and C ^ o v S ^ p 1 , S ^ p 2 = ε ^ 2 / n 2 . All of the other covariances are zero, since the two samples are independent. The estimators of ρ 1 and ρ 2 are ρ ^ 1 = n 1 n 111 n 11 · n 1 · 1 n 11 · n 1 n 11 · n 1 · 1 n 1 n 1 · 1 and ρ ^ 2 = n 2 n 200 n 20 · n 2 · 0 n 20 · n 2 n 20 · n 2 · 0 n 2 n 2 · 0 . Assuming that the disease prevalence p is known, the estimators of the predictive values are:
P P V 1 ^ = p n 2 n 11 · p n 2 n 11 · + q n 1 n 2 n 20 ·   and   N P V 1 ^ = q n 1 n 20 · p n 2 n 1 n 11 · + q n 1 n 20 · ,
for Test 1, and
P P V 2 ^ = p n 2 n 1 · 1 p n 2 n 1 · 1 + q n 1 n 2 n 2 · 0   and   N P V 2 ^ = q n 1 n 2 · 0 p n 2 n 1 n 1 · 1 + q n 1 n 2 · 0
for Test 2, where q = 1 p . Let the variance-covariance matrixes be defined as:
S ^ e = V a r S ^ e 1 C o v S ^ e 1 , S ^ e 2 C o v S ^ e 1 , S ^ e 2 V a r S ^ e 2
and
S ^ p = V a r S ^ p 1 C o v S ^ p 1 , S ^ p 2 C o v S ^ p 1 , S ^ p 2 V a r S ^ p 2 .
Let θ = S e 1 , S e 2 , S p 1 , S p 2 T be a vector whose components are the sensitivities and the specificities, and let ω = P P V 1 , P P V 2 , N P V 1 , N P V 2 T be a vector whose components are the PVs. The variance-covariance matrix of θ ^ is:
θ ^ = 1 0 0 0 S ^ e + 0 0 0 1 S ^ p ,
where is the Kronecker product. Applying the delta method, the matrix of variances- covariances of ω ^ is:
ω ^ = ω θ θ ^ ω θ T .
Expressions of the variances-covariances of the PVs can be seen in Appendix A. The PVs of each BDT depend on the same parameters, the sensitivity and the specificity of the test and disease prevalence, and therefore they are related parameters. Consequently, the PVs of the two BDTs can be compared simultaneously. The global hypothesis test to simultaneously compare the PVs of the two BDTs is:
H 0 : P P V 1 = P P V 2   and   N P V 1 = N P V 2 H 1 : at   least   one   equality   is   not   true ,
which is equivalent to the hypothesis test:
H 0 : A ω = 0   vs   H 1 : A ω 0
where A is a complete range matrix sized 2 × 4 whose elements are known constants, i.e.:
A = 1 1 0 0 0 0 1 1 .
As the vector ω ^ is distributed asymptotically according to a multivariate normal distribution, i.e., n 1 + n 2 ω ^ ω n 1 + n 2 N 0 , Σ ω , then the test statistic for the global hypothesis test (4) is:
Q 2 = ω ^ T A T A ^ ω ^ A T 1 A ω ^ ,
which is distributed asymptotically according to Hotelling’s T-squared distribution with a dimension 2 and n 1 + n 2 degrees of freedom, where 2 is the dimension of the vector A ω ^ . When n 1 + n 2 is large, the statistic Q 2 is approximately distributed according to a central chi-square distribution with 2 degrees of freedom when the null hypothesis is true.
On the other hand, the individual comparison of the positive (negative) predictive values is solved with the hypothesis test:
H 0 : P V 1 = P V 2   vs   H 0 : P V 1 P V 2 ,
where PV is PPV or NPV. Based on the asymptotic normality of the estimators, the test statistic for this hypothesis test is:
z = P V 1 ^ P V 2 ^ V ^ a r P V 1 ^ + V ^ a r P V 2 ^ 2 C ^ o v P V 1 ^ , P V 2 ^ ,
which is distributed asymptotically according to a normal standard distribution, and where the variances-covariances are obtained from the Equation (3) (see Appendix A).
The global hypothesis test H 0 : A ω = 0 simultaneously compares the PPVs and the NPVs of the two BDTs. Some alternative methods to this global hypothesis test, based on the individual hypothesis tests, are: (1) testing the hypotheses H 0 : P P V 1 = P P V 2 and H 0 : N P V 1 = N P V 2 (Equation (6)) each one to an α error; (2) testing the hypotheses H 0 : P P V 1 = P P V 2 and H 0 : N P V 1 = N P V 2 (Equation (6)) and applying a multiple comparison method such as the Bonferroni method [12] or the Holm method [13], which are methods that are very easy to apply based on the p-values. Bonferroni method [12] consists of solving each individual hypothesis test to an error equal to α / 2 . The Holm method is a step-down method which is based on Bonferroni method but is more conservative. In Appendix B, the Holm method [13] is summarized.

3. Simulation Experiments

Simulation experiments were carried out to study the type I errors and the powers of the four methods proposed to simultaneously compare the predictive values: the global hypothesis test based on the chi-square distribution (Equation (5)), the individual hypothesis tests each one to an α error (Equation (6)), the individual hypothesis tests (Equation (6)) applying the Bonferroni method and the individual hypothesis tests (Equation (6)) applying the Holm method. We have also studied the effect of a misspecification of the prevalence on the asymptotic behavior of these methods and on the estimators of the PVs.
The experiments were designed setting the values of the PVs. For each BDT, we took as PVs the values 0.60, 0.65, …, 0.90, 0.95, and as disease prevalence we took the values 10%, 25% and 50%. Based on the PVs and the prevalence, Se and Sp of each BDT were calculated from the Equation (1), only considering those cases in which the solutions are between 0 and 1. As values of the correlation coefficients ρ 1 and ρ 2 we took low values (25% of the maximum value), intermediate (50% of the maximum value) and high (75% of the maximum value), where the maximum value of each correlation coefficient is: max ρ 1 = min S e 1 1 S e 2 , 1 S e 1 S e 2 S e 1 1 S e 1 S e 2 1 S e 2 and max ρ 2 = min S p 1 1 S p 2 , 1 S p 1 S p 2 S p 1 1 S p 1 S p 2 1 S p 2 respectively. As sample sizes, we took the values n i = 50 , 75 , 100 , 200 , 500 . The simulation experiments were carried out with R [14], using the “bindata” package [15] to generate the samples of each type I bivariate binomial distribution.
Regarding the random samples, these were generated in the following way. Firstly, once the values of the PVs and of the prevalence were set, we calculated the sensitivities, the specificities and the maximum values of the coefficients ρ 1 and ρ 2 . We then generated 10,000 random samples from a type I bivariate binomial distribution with a sample size n 1 , probabilities S e 1 and S e 2 , and correlation coefficient ρ 1 . Similarly, we generated another 10,000 random samples from a type I bivariate binomial distribution with a sample size n 2 , probabilities S p 1 and S p 2 , and correlation coefficient ρ 2 . In this way, we obtained the marginal frequencies n 11 · and n 1 · 1 ( n 20 · and n 2 · 0 ) of each one of the 10,000 case (control) samples. The rest of the marginal frequencies were easily calculated: n 10 · = n 1 n 11 · , n 1 · 0 = n 1 n 1 · 1 , n 21 · = n 2 n 20 · and n 2 · 1 = n 2 n 2 · 0 . In order to construct the 2 × 2 table of each case sample, we generated a random value n 111 from a doubly truncated binomial distribution of parameters n 1 and ξ 111 = S e 1 S e 2 + ε 1 , with n 11 · + n 1 · 1 n 1 n 111 min n 11 · , n 11 · . This is necessary so that the sum of the frequencies leads to the marginal totals randomly generated through the type I bivariate binomial distribution. In the same way, in order to construct the 2 × 2 table of each control sample, we generated a random value n 200 from a doubly truncated binomial distribution of parameters n 2 and ξ 200 = S p 1 S p 2 + ε 2 , with n 20 · + n 2 · 0 n 2 n 200 min n 20 · , n 2 · 0 . For each one of the 10,000 case (control) samples, once we have generated the values n 11 · , n 1 · 1 and n 111 ( n 20 · , n 2 · 0 and n 200 ) it is easy to construct the complete 2 × 2 table. Thus, n 110 = n 1 n 11 · , n 101 = n 1 · 1 n 111 and n 100 = n 10 · n 101 for the case samples, and n 201 = n 20 · n 200 , n 210 = n 2 · 0 n 200 and n 211 = n 21 · n 210 for the control samples. For the experiments α = 5 % was set. Moreover, all of the samples were generated in such a way that in all of them the parameters and the variances-covariances can be estimated. If in a random sample it is obtained that n i 10 = n i 01 = 0 , with i = 1 , 2 , then S ^ e i = S ^ p i = 1 and V ^ a r S ^ e i = V ^ a r S ^ p i = 0 , and therefore the test statistic Q 2 = ω ^ T A T A ^ ω ^ A T 1 A ω ^ cannot be calculated since A ^ ω ^ A T is a non-singular matrix. This problem occurs mainly when the sample size is small or moderate. In this situation, the sample has been discarded and another is generated in its place until the 10,000 samples are obtained.

3.1. Type I Errors and Powers

In Table 2 and Table 3, we can see some results obtained for the type I errors of the global test and of the alternative methods proposed in Section 2. In these tables, we can only see the results for the global test, the individual comparisons with α = 5 % and with the Bonferroni method. The results obtained with the Holm method are not shown as they are practically the same as those obtained with the Bonferroni method. From the results obtained we can draw the following conclusions. In general terms, the type I error of the global hypothesis test fluctuates around the nominal error, especially in the case of samples sized n i 100 , depending on the prevalence and the correlations between the two BDTs. For samples with smaller sizes n i 75 , the type I error of the global test is lower than α = 5 % . The correlations between the two BDTs have an important effect on the type I error of the global test, with a decrease in the type I error when there is an increase in the correlation coefficients.
Regarding the method based on the individual hypothesis tests H 0 : P P V 1 = P P V 2 and H 0 : N P V 1 = N P V 2 to an error α = 5 % each one of them, the type I error may clearly overwhelm the nominal error (a situation that we have considered when the type I error is greater than 7%), especially when the correlations are not high. Consequently, this method may lead to erroneous results (false significances) and, therefore, should not be used. As for solving the global test from the individual tests applying the Bonferroni (Holm) method, the type I error has a very similar behavior to that of the global hypothesis test.
Regarding the powers of the hypothesis tests, in Table 4 and Table 5 we can see some of the results obtained for the global test and other alternative methods. The results obtained with the Holm method are not shown as they are practically the same as those obtained with the Bonferroni method. The power of the global hypothesis test is calculated as the proportion of samples in which it is accepted that P P V 1 P P V 2 or N P V 1 N P V 2 (being true that P P V 1 P P V 2 or N P V 1 N P V 2 ). From the results, the following conclusions are obtained. The disease prevalence has an important effect on the power of each one of the methods to solve the global test, and the power increases with an increase in the prevalence. Regarding the correlations ρ 1 and ρ 2 , these do not have a clear effect on the power, and the power increases sometimes and decreases other times when the correlations increase. In general terms, when the prevalence is small p = 10 % we need large samples n i > 500 so that the power of the global hypothesis test is greater than 80%; for a prevalence of 25% with sample sizes n i 200 we obtain a power greater than 80%; and for a very large prevalence p = 50 % with sample sizes n i 50 we obtain a very higher power, greater than 80%–90%, depending on the difference between the PVs.
The power of the method based on the individual hypothesis tests to an error α = 5 % is greater than that of the global test based on the chi-square distribution due to the fact that its type I error is also greater. Regarding the hypothesis test based on the individual tests with the Bonferroni method, in general terms, its power is very similar to that of the global test when the sample sizes are large. When the sample sizes are small or moderate, in general terms and depending on prevalence and correlations, the power of the global test is slightly greater than that of the individual tests with the Bonferroni method. The same conclusions are obtained when the Holm method is applied (whose results are almost identical to those of the Bonferroni method).
The graphs in Figure 1 show the powers of the three methods when P P V 1 = 0.90 , N P V 1 = 0.80 , 0.85 , 0.90 , 0.95 , P P V 2 = 0.85 and N P V 2 = 0.90 , for different sample sizes n 1 = n 2 = 50 , 100 , 200 , p = 25 % , 50 % and values intermediate of the correlation coefficients. These graphs show that when N P V 1 varies and the rest of the PVs are constant, the powers decrease when the prevalence increases. Similarly, the graphs in Figure 2 show the powers of the three methods when P P V 1 = 0.80 , 0.85 , 0.90 , 0.95 , N P V 1 = 0.95 , P P V 2 = 0.60 and N P V 2 = 0.95 , for different sample sizes n 1 = n 2 = 50 , 100 , 200 , p = 10 % , 25 % and values intermediate of the correlation coefficients. These graphs show that when the P P V 1 varies and the rest of the PVs are constant, the power of each method increases when the prevalence increases.
As conclusions of the results obtained in the simulation experiments, the global hypothesis test based on the chi-square distribution behaves well in terms of the type I error (it does not overwhelm the nominal error of 5%), the same as the individual tests along with the Bonferroni (Holm) method. The method based on the individual tests to a global error α = 5 % should not be used as it may clearly overwhelm the nominal error.
In the simulation experiments, the proportion of times that P P V 1 P P V 2 and that N P V 1 N P V 2 are correctly concluded has also been studied. This issue is of special interest when the alternative hypothesis of the global test is true, as it can be a valid method to investigate the causes of significance. The study was carried out by applying the individual hypothesis tests together with the Bonferroni (Holm) method. Individual tests to an α error have not been considered as they have a type I error that can exceed the nominal error. If it is verified that P V 1 P V 2 , then this study is equivalent to studying the power of the individual test H 0 : P V 1 = P V 2 to an α / 2 error (since the Bonferroni method has been applied), where P V i is P P V i or N P V i . If it is verified that P V 1 = P V 2 then this study is equivalent to studying the type I error of the individual test H 0 : P V 1 = P V 2 to an α / 2 error. In the scenarios considered in Table 4 and Table 5 it is verified that P P V 1 P P V 2 and that N P V 1 = N P V 2 . Therefore, for these two scenarios, the power of the test H 0 : P P V 1 = P P V 2 and the type I error of the test H 0 : N P V 1 = N P V 2 have been studied, each with an error equal to α / 2 = 2.5 % . Table 6 and Table 7 show the results obtained applying the Bonferroni method. The results obtained with the Holm method are not shown as they are practically the same as those obtained with the Bonferroni method.
In general terms, the hypothesis test H 0 : P P V 1 = P P V 2 has a high power when the sample sizes are moderate or high, depending on the prevalence and the correlation coefficients. Its behavior is very similar to that of the global hypothesis test. With respect to the test H 0 : N P V 1 = N P V 2 , its type I error fluctuates around the nominal error (2.5%) when the sample sizes are moderate or large, depending on the prevalence of the correlation coefficients. In general terms, the hypothesis tests H 0 : P P V 1 = P P V 2 and H 0 : N P V 1 = N P V 2 have a good asymptotic behavior, both in terms of power and type I error.
From the results obtained in the simulation experiments, we propose the following method to compare the PVs of two BDTs subject to a case-control design: (1) Applying the global hypothesis test based on the chi-square distribution (Equation (5)) to an α error; (2) If the global hypothesis test is not significant, the equality hypothesis of the PVs is not rejected; if the global hypothesis test is significant to an α error, the investigation of the causes of the significance is made by testing the individual tests (Equation (6)) and applying the Bonferroni method or the Holm method to an α error. Therefore, if the global test is significant, the investigation of the significance consists in solving the individual hypothesis tests H 0 : P P V 1 = P P V 2 and H 0 : N P V 1 = N P V 2 , each of them to an α / 2 error (Bonferroni method) or applying Holm method.
This method to simultaneously compare the PVs is very similar to other methods used in other statistical models, such as the analysis of variance: first the global test is resolved to an α error, and if it is significant then the causes of significance are investigated from pairwise comparisons and the application of a multiple comparison method.

3.2. Effect of the Prevalence

The estimation and comparison of the PVs of two BDTs subject to a case-control design requires knowledge of the disease prevalence. To study the effect of a misspecification of the prevalence on the comparison of the PVs and on the estimators of the PVs, we carried out simulation experiments similar to those made to study the type I errors and the powers. For this purpose, we took as the prevalence for the inference a misspecification equal to 10% and to 20% of the value of the prevalence set, and we have studied the type I errors and the powers of the global test and of the Bonferroni and Holm methods, and the relative root mean square error (RRMSE) of the estimator of each PVs. Thus, for each estimator we calculated the relative root mean square error (RRMSE), i.e.,
R R M S E P V ^ i = 1 N k = 1 N P V ^ i k P V i 2 P V i ,
where P V i is the PPV or the NPV of the ith BDT i = 1 , 2 and P V ^ i k is its estimator calculated from the kth sample k = 1 , , N , and N = 10 , 000 . For the values of the parameters we took as prevalence p = 10 % , 25 % , 50 % respectively, and to estimate the PVs we took as prevalence p = p ± d × p with d = 10 % , 20 % . A value d = 10 % d = 20 % can be considered as a small (moderate) value of the relative deviation.
Table 8 shows some of the results obtained for the type I errors and the powers of the global test and the Bonferroni method (the results of the Holms method are not shown as they are practically identically to those obtained with the Bonferroni method). In this Table we show the results when there is no misspecification of the prevalence p = p and when there is a misspecification of the prevalence ( p < p or p > p ). From the results of these experiments, it is verified that the type I errors of the methods studied do not overwhelm the nominal error ( α = 5 % ). In general terms there are no important differences between the type I errors when there is a misspecification of the prevalence and when there is not. Regarding the powers, the conclusions are also very similar, there are no important differences between the powers when there is a misspecification of the prevalence and when there is not.
Regarding the estimators, Table 8 shows some of the results obtained for the RRMSEs (in %) of the estimators of the PVs of Test 1 (the results for Test 2 are identical). In general terms, the difference between the RRMSEs is small (around 5% or less, in absolute value) when the two sample sizes are moderate n i = 100 or large n i 200 and the relative deviation is small (10%) or moderate (20%). Therefore, a small or moderate misspecification of the prevalence ( p < p or p > p ) does not have an important effect on the estimators of the PVs when the samples are moderate or large. Additionally, there is not an important difference between the RRMSEs when the sample sizes are small n i 75 and the relative deviation is small. However, the difference between the RRMSEs is larger when the sample sizes are small and the relative deviation is moderate. In this situation, a misspecification of the prevalence has an important effect on the estimators of the PVs.

4. Example

The results obtained have been applied to the diagnosis of coronary heart disease, using an electrocardiogram and an echocardiography as diagnostic tests. Both tests have been applied to a sample of 105 older men with coronary heart disease (case sample) and to another sample of 120 older men without this disease (control sample). In Table 9 we can see the frequencies obtained, where the random variable T 1 models the result of the electrocardiogram and the variable T 2 models the result of the echocardiography. In order to illustrate the method proposed, we are going to consider that the prevalence of the disease in older men is 5%. The objective is to compare the clinical accuracy (PVs) of both BDTs in the population whose prevalence of coronary heart disease is 5%. The comparison of the PVs will be made with α = 5 % .
From the case sample, the estimates of the two sensitivities (and their standard errors, SE) and of the correlation coefficient between them are S ^ e 1 ± S E = 0.829 ± 0.037 , S ^ e 1 ± S E = 0.790 ± 0.040 and ρ ^ 1 = 0.511 . From the control sample, the estimates of the two specificities and of the correlation coefficient between them are S ^ p 1 ± S E = 0.950 ± 0.020 , S ^ e 1 ± S E = 0.858 ± 0.032 and ρ ^ 2 = 0.345 . Assuming that the prevalence of coronary heart disease is 5%, the estimates of the PVs are:
P P V ^ 1 = 0.05 × 0.829 0.05 × 0.829 + 1 0.05 × 1 0.950 = 0.466 , P P V ^ 2 = 0.05 × 0.790 0.05 × 0.790 + 1 0.05 × 1 0.858 = 0.227 , N P V ^ 1 = 1 0.05 × 0.950 0.05 × 1 0.829 + 1 0.05 × 0.950 = 0.991
and
N P V ^ 2 = 1 0.05 × 0.858 0.05 × 1 0.790 + 1 0.05 × 0.858 = 0.987 .
Applying the delta method (Equation (3)), the estimated variance-covariance matrix of the estimators of the PVs is:
^ ω ^ = 0.009926 0.001398 0.000041 0.000029 0.001398 0.001632 0.000012 0.000039 0.000041 0.000012 0.000004 0.000002 0.000029 0.000039 0.000002 0.000006 .
Applying Equation (5), the value of the test statistic for the global test:
H 0 : P P V 1 = P P V 2   and   N P V 1 = N P V 2 H 1 : at   least   one   equality   is   not   true ,
is Q 2 = 7.516 and the p-value is 0.023, and therefore the null hypothesis of the global test is rejected. Testing the individual hypothesis tests it is found that the value of the test statistic for H 0 : P P V 1 = P P V 2 is equal to 2.552 (two sided p-value is 0.011) and that the value of the test statistic for H 0 : N P V 1 = N P V 2 is equal to 1.469 (two sided p-value is 0.142). Applying the Bonferroni (or Holm) method, the hypothesis of equality of the positive predictive values is rejected and the hypothesis of equality of the negative predictive values is not rejected. Therefore, in a population in which the prevalence of coronary heart disease is 5%, the positive predictive value of electrocardiogram is significantly greater than that of the echocardiography (95% confidence interval for the difference: 0.056 to 0.422), while there are no significant differences between the two negative predictive values.

5. More Than Two BDTs

Let us consider that J BDTs J 3 are applied to all of the individuals in the case sample and the control sample. For each BDT we define the random variable T j in a similar way to how this was done in Section 2. Let S e j and S p j be the sensitivity and the specificity of the jth BDT, with j = 1 , , J . Let n 1 i 1 i J be the number of individuals with the disease for whom T 1 = i 1 ,…, T J = i J , with i j = 1 when the result of the jth BDT is positive and i j = 0 when it is negative. In a similar way, n 2 i 1 i J is the number of without the disease for whom T 1 = i 1 ,…, T J = i J . Let us consider the probabilities ξ h i 1 , , i J = P T 1 = i 1 , T 2 = i 2 , , T J = i J , with h = 1 , 2 . Thus, for example for three BDTs, using the dependence model of Torrance–Rynard and Walter [16], these probabilities are:
ξ 1 i 1 i 2 i 3 = j = 1 3 S e j i j 1 S e j 1 i j + j , k , j < k 3 1 i j i k ε 1 j k
and
ξ 2 i 1 i 2 i 3 = j = 1 3 S p j 1 i j 1 S p j i j + j , k , j < k 3 1 i j i k ε 2 j k ,
with i j = 0 , 1 , i k = 0 , 1 and j , k = 1 , 2 , 3 , and where ε 1 j k ( ε 2 j k ) is the covariance between the jth BDT and the kth BDT for individuals with the disease (without the disease). The estimators of these probabilities are ξ ^ h i 1 i J = n h i 1 i J / n h , with h = 1 , 2 . The sensitivity and the specificity of the jth BDT are:
S e j = i 1 , , i J = 0 i j = 1 1 ξ 1 i 1 , , i J   and   S p j = i 1 , , i J = 0 i j = 0 1 ξ 2 i 1 , , i J ,
and its estimators are:
S ^ e j = i 1 , , i J = 0 i j = 1 1 n 1 i 1 , , i J n 1   and   S ^ p j = i 1 , , i J = 0 i j = 0 1 n 2 i 1 , , i J n 2 .
The estimators of the variances-covariances of these estimators are V ^ a r S ^ e j = S ^ e j 1 S ^ e j / n 1 , V ^ a r S ^ p j = S ^ p j 1 S ^ p j / n 2 , C ^ o v S ^ e j , S ^ e k = ε ^ 1 j k / n 1 and C ^ o v S ^ p j , S ^ p k = ε ^ 2 j k / n 2 , and the rest of the covariances are equal to zero. The estimators of the PVs of the jth BDT are:
P P V j ^ = p n 2 i 1 , , i J = 0 i j = 1 1 n 1 i 1 , , i J p n 2 i 1 , , i J = 0 i j = 1 1 n 1 i 1 , , i J + q n 1 i 1 , , i J = 0 i j = 1 1 n 2 i 1 , , i J
and
N P V j ^ = q n 1 i 1 , , i J = 0 i j = 0 1 n 2 i 1 , , i J q n 1 i 1 , , i J = 0 i j = 0 1 n 2 i 1 , , i J + p n 2 i 1 , , i J = 0 i j = 0 1 n 1 i 1 , , i J ,
where p is the disease prevalence and q = 1 p .
Let θ = S e 1 , , S e J , S p 1 , , S p J T be the vector whose components are the sensitivities and the specificities, and let ω = P P V 1 , , P P V J , N P V 1 , , N P V J T be the vector whose components are the PVs. The variance-covariance matrix of the vector θ ^ , with a dimension 2 J × 2 J , is similar to that given in Equation (2), where S ^ e and S ^ p are matrixes with a dimension J × J . Applying the delta method, the variance-covariance matrix of ω ^ , with a dimension 2 J × 2 J , has an expression similar to that given in Equation (3). The PVs of each one of the J BDTs depend on the same parameters (the sensitivity and the specificity of the jth diagnostic test) and, therefore, these parameters can be compared simultaneously. The global hypothesis test to simultaneously compare the PVs of the J BDTs is:
H 0 : P P V 1 = P P V 2 = = P P V J   and   N P V 1 = N P V 2 = = N P V J H 1 : at   least   one   equality   is   not   true ,
which is equivalent to the hypothesis test:
H 0 : A ω = 0   vs   H 1 : A ω 0 ,
where A is a matrix with a dimension 2 J 1 × 2 J , i.e.,
A = A 1 A 0 A 0 A 1 .
A 0 is a matrix with a dimension J 1 × J whose elements are all equal to 0, and A 1 is a matrix with a dimension J 1 × J where each component i , i is equal to 1, each element i , i + 1 is equal to 1 for i = 1 , , J 1 , and the rest of the elements in this matrix are equal to 0. Applying the multivariate central limit theorem it is verified that n 1 + n 2 ω ^ ω n 1 + n 2 N 2 J 0 , Σ ω . Then, the statistic Q 2 = ω ^ A T A ^ ω ^ A T 1 A ω ^ is distributed according to Hotelling’s T-squared distribution with a dimension 2 J 1 and n 1 + n 2 degrees of freedom, where 2 J 1 is the dimension of the vector A ω ^ . When n 1 + n 2 is large, the statistic Q 2 is distributed according to a central chi-squared distribution with 2 J 1 degrees of freedom when the null hypothesis is true, i.e.,
Q 2 = ω ^ A T A ^ ω ^ A T 1 A ω ^ n χ 2 J 1 2 .
Finally, the method to compare the PVs of the J BDTs would consist of the following steps: (1) Solve the global hypothesis test to an α error calculating the statistic Q 2 = ω ^ A T A ^ ω ^ A T 1 A ω ^ based on the chi-squared distribution; (2) if the global test is not significant to an α error then we do not reject the homogeneity of the J PVs, but if the hypothesis test is significant then the causes of significance are investigated comparing the PPVs (NPVs) in pairs (Equation (6)) and applying a multiple comparison method (e.g., Bonferroni or Holm).

6. Discussion

The comparison of the positive and negative predictive values of two binary diagnostic tests is an important topic in the study of statistical methods in diagnostic medicine. Subject to a paired design, this topic has been subject to different studies. In this article we studied the simultaneous comparison of the predictive values of two diagnostic tests subject to a case-control design, analyzing and comparing several methods. These methods consisted of a global test based on the chi-square distribution, a method based on the individual comparisons each one to an α error, and other two methods based on individual comparisons along with a multiple comparison method. The multiple comparison methods that were used were the Bonferroni method and the Holm method, which are methods based on the p-values of the individual hypothesis tests and are very easy to apply. The methods studied to compare the predictive values require knowing the prevalence of the disease. The prevalence can be known from other studies, such as population studies of health services, cohort studies, etc. If the researcher has a great uncertainty about the value of the prevalence, the problem can be solved by using several values for the prevalence and then analyzing and comparing the results obtained.
Simulation experiments were carried out to study the type I errors and the powers of the four methods proposed. These experiments were based on the generation of random samples with type I bivariate binomial distributions, which are the distributions that are inherent to case-control design, since proportions of marginal totals are estimated from these samples. The results have shown that the global hypothesis test based on the chi-square distribution behaves well in terms of type I error, and does not overwhelm the nominal error. Regarding its power, in general this strongly depends on the disease prevalence, and it is necessary to have very large samples when the prevalence is small and relatively small sample sizes when the prevalence is high, so that the power will be high.
Based on the results of the simulation experiments, a method has been proposed to compare the predictive values of two diagnostic tests subject to a case-control design. This method, which is similar to that proposed by Roldán-Nofuentes et al. [8], consists of the following steps: (1) Simultaneously comparing the predictive values applying the global hypothesis test based on the chi-square distribution to an α error; (2) if the global hypothesis test is not significant, then the equality hypothesis of the PVs is not rejected. If the global hypothesis test is significant to an α error, then the causes of the significance are studied solving the individual hypothesis tests and applying the Bonferroni method or the Holm method to an α error. This procedure that we propose is similar to the analysis of variance: firstly, the global test is solved and, if this is significant, then the causes of the significance are studied starting with paired comparisons along with some multiple comparison method.
Simulation experiments were carried out to study the effect of a misspecification of the prevalence in the asymptotic behavior of the global hypothesis test based on the chi-square distribution and on the methods based on multiple comparisons. In general terms, we can conclude that a small or moderate misspecification of the prevalence do not have an important effect on the behavior of these hypothesis tests, especially when the sample sizes are moderate or large.
The global hypothesis test was extended to the situation in which we simultaneously compare the PVs of more than two BDTs, and for this we propose a method which is similar to that proposed for two BDTs. To be able to calculate the global test statistic Q 2 = ω ^ A T A ^ ω ^ A T 1 A ω ^ it is necessary that A ^ ω ^ A T to be non-singular. For two BDTs, this matrix is non-singular when it is verified that n i 10 + n i 01 > 0 , with i = 1 , 2 . If n i 10 = n i 01 = 0 then the method proposed to compare the PVs cannot be applied A solution to this problem consists in adding the value 0.5 to all the observed frequencies (the sample size increases by two units), a very frequent solution in the analysis of 2 × 2 tables. Simulation experiments have been carried out to study the type I errors and the powers of the hypothesis tests proposed in Section 2, using this solution when in a sample it is verified that n i 10 = n i 01 = 0 . These experiments have been designed in a similar way to those performed in Section 3. Table 10 shows some results (type I errors and powers) for some of the scenarios considered in Section 3, as well as the average proportions (of the three correlation scenarios) of case (control) samples in which the value 0.5 has been added. Obviously, the proportion of samples in which the value 0.5 has been added is greater when the sample size is small. In general terms, the conclusions are the same as those obtained in the simulation experiments presented in Section 3, although the powers of all hypothesis tests are slightly lower when the sample sizes are small. Therefore, adding 0.5 to all the observed frequencies of a sample in which n i 10 = n i 01 = 0 is an adequate solution to be able to apply the PVs comparison method.

Author Contributions

The two authors have collaborated equally in the realization of this work. Both authors have read and agreed to the published version of the manuscript.

Funding

This research was supported by the Spanish Ministry of Economy, Grant Number MTM2016-76938-P, and by the University of Nouakchott Alaasriya.

Institutional Review Board Statement

Not applicable.

Informed Consent Statement

Not applicable.

Data Availability Statement

Not applicable.

Acknowledgments

We thank the three referees for their helpful comments that improved the quality of this manuscript. We thank Professor Cheikh Saad Bouh Camara, the president of the University of Nouakchott, for his help to finance the publication of this manuscript.

Conflicts of Interest

The authors declare no conflict of interest.

Appendix A

Performing algebraic operations in Equation (3) it is found that:
V a r P P V 1 ^ = p 2 S e 1 + p Q 1 Q 1 2 2 V a r S ^ e 1 + p q S e 1 Q 1 2 2 V a r S ^ p 1 ,
V a r P P V 2 ^ = p 2 S e 2 + p Q 2 Q 2 2 2 V a r S ^ e 2 + p q S e 2 Q 2 2 2 V a r S ^ p 2 ,
V a r N P V 1 ^ = p q S p 1 1 Q 1 2 2 V a r S ^ e 1 + q 1 Q 1 q 2 S p 1 1 Q 1 2 2 V a r S ^ p 1 ,
V a r N P V 2 ^ = p q S p 2 1 Q 2 2 2 V a r S ^ e 2 + q 1 Q 2 q 2 S p 2 1 Q 2 2 2 V a r S ^ p 2 ,
C o v P P V 1 ^ , P P V 2 ^ = p Q 1 p 2 S e 1 Q 1 2 p Q 2 p 2 S e 2 Q 2 2 C o v S ^ e 1 , S ^ e 2 + p 2 q 2 S e 1 S e 2 Q 1 2 Q 2 2 C o v S ^ p 1 , S ^ p 2 ,
C o v P P V 1 ^ , N P V 1 ^ = p q Q 1 2 1 Q 1 2 p Q 1 p 2 S e 1 S p 1 V a r S ^ e 1 + q 1 Q 1 q 2 S p 1 S e 1 V a r S ^ p 1 ,
C o v P P V 1 ^ , N P V 2 ^ = p q p Q 1 p 2 S e 1 Q 1 2 1 Q 2 2 S p 2 C o v S ^ e 1 , S ^ e 2 + p q q 1 Q 2 q 2 S p 2 Q 1 2 1 Q 2 2 S e 1 C o v S ^ p 1 , S ^ p 2 ,
C o v P P V 2 ^ , N P V 1 ^ = p q Q 2 2 1 Q 1 2 p Q 2 p 2 S e 2 S p 1 C o v S ^ e 1 , S ^ e 2 + q 1 Q 1 q 2 S p 1 S e 2 C o v S ^ p 1 , S ^ p 2 ,
C o v P P V 2 ^ , N P V 2 ^ = p q Q 2 2 1 Q 2 2 p Q 2 p 2 S e 2 S p 2 V a r S ^ e 2 + q 1 Q 2 q 2 S p 2 S e 2 V a r S ^ p 2
and
C o v N P V 1 ^ , N P V 2 ^ = p 2 q 2 S p 1 S p 2 1 Q 1 2 1 Q 2 2 C o v S ^ e 1 , S ^ e 2 + q 1 Q 1 q 2 S p 1 1 Q 1 2 q 1 Q 2 q 2 S p 2 1 Q 2 2 C o v S ^ p 1 , S ^ p 2 ,
and where q = 1 p and Q i = p × S e i + q × 1 S p i .

Appendix B

Let us assume that we are going to solve K hypothesis test H 0 k vs. H 1 k with k = 1 , , K . Let p 1 p 2 p K be the p-values obtained ordered from the lowest to the highest, and therefore p k is the p-value that corresponds to the hypothesis test H 0 k vs. H 1 k . The Holm method [13] consists of the following steps:
Step 1. If p 1 α / K then hypothesis H 0 1 is rejected and we go to the next step; if p 1 > α / K then no null hypothesis is rejected and the process finishes.
Step 2. If p 2 α / K 1 then H 0 2 is rejected and we go to the next step; if p 2 > α / K 1 we do not reject the null hypotheses H 0 k with k = 2 , , K and the process finishes….
Step K. If p K α then H 0 K is rejected and the process finishes; and if p K > α then H 0 K is not rejected and the process finishes.

References

  1. Pepe, M.S. The Statistical Evaluation of Medical Tests for Classification and Prediction; Oxford University Press: New York, NY, USA, 2003. [Google Scholar]
  2. Bennett, B.M. On comparison of sensitivity, specificity and predictive value of a number of diagnostic procedures. Biometrics 1972, 28, 793–800. [Google Scholar] [CrossRef] [PubMed]
  3. Bennett, B.M. On tests for equality of predictive values for t diagnostic procedures. Stat. Med. 1985, 4, 535–539. [Google Scholar] [CrossRef] [PubMed]
  4. Leisenring, W.; Alonzo, T.; Pepe, M.S. Comparisons of predictive values of binary medical diagnostic tests for paired designs. Biometrics 2000, 56, 345–351. [Google Scholar] [CrossRef] [PubMed]
  5. Wang, W.; Davis, C.S.; Soong, S.J. Comparison of predictive values of two diagnostic tests from the same sample of subjects using weighted least squares. Stat. Med. 2006, 25, 2215–2229. [Google Scholar] [CrossRef] [PubMed]
  6. Kosinski, A.S. A weighted generalized score statistic for comparison of predictive values of diagnostic tests. Stat. Med. 2013, 32, 964–977. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  7. Moskowitz, C.S.; Pepe, M.S. Comparing the predictive values of diagnostic tests: Sample size and analysis for paired study designs. Clin. Trials 2006, 3, 272–279. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  8. Roldán-Nofuentes, J.A.; Luna del Castillo, J.D.; Montero-Alonso, M.A. Global hypothesis test to simultaneously compare the predictive values of two binary diagnostic tests. Comput. Stat. Data Anal. 2012, 56, 1161–1173. [Google Scholar] [CrossRef]
  9. Mercaldo, N.D.; Kit, F.L.; Zhou, X.H. Confidence intervals for predictive values with an emphasis to case-control studies. Stat. Med. 2007, 26, 2170–2183. [Google Scholar] [CrossRef] [PubMed]
  10. Vacek, P.M. The effect of conditional dependence on the evaluation of diagnostic tests. Biometrics 1985, 41, 959–968. [Google Scholar] [CrossRef] [PubMed]
  11. Kocherlakota, S.; Kocherlakota, K. Bivariate Discrete Distributions; Marcel Dekker INC: New York, NY, USA, 1992. [Google Scholar]
  12. Bonferroni, C.E. Teoria statistica delle classi e calcolo delle probabilità. Pubblicazioni del R Istituto Superiore di Scienze Economiche e Commerciali di Firenze 1936, 8, 3–62. [Google Scholar]
  13. Holm, S. A simple sequential rejective multiple testing procedure. Scand. J. Stat. 1979, 6, 65–70. [Google Scholar]
  14. RC Team. R: A Language and Environment for Statistical Computing; RC Team: Vienna, Austria, 2016; Available online: https://www.R-project.org/ (accessed on 17 March 2021).
  15. Leisch, F.; Weingessel, A.; Hornik, K. Bindata Package. Available online: https://cran.r-project.org/web/packages/bindata/ (accessed on 17 March 2021).
  16. Torrance-Rynard, V.L.; Walter, S.D. Effects of dependent errors in the assessment of diagnostic test performance. Stat. Med. 1997, 16, 2157–2175. [Google Scholar] [CrossRef]
Figure 1. Powers of the three methods when the negative predictive value (NPV) of a binary diagnostic test varies and the rest of the PVs are constant.
Figure 1. Powers of the three methods when the negative predictive value (NPV) of a binary diagnostic test varies and the rest of the PVs are constant.
Mathematics 09 00658 g001
Figure 2. Powers of the three methods when the positive predictive value (PPV) of a binary diagnostic test varies and the rest of the PVs are constant.
Figure 2. Powers of the three methods when the positive predictive value (PPV) of a binary diagnostic test varies and the rest of the PVs are constant.
Mathematics 09 00658 g002
Table 1. Probabilities and observed frequencies subject to case-control design.
Table 1. Probabilities and observed frequencies subject to case-control design.
Probabilities
CaseControl
T 2 = 1 T 2 = 0 Total T 2 = 1 T 2 = 0 Total
T 1 = 1 ξ 111 ξ 110 S e 1 T 1 = 1 ξ 211 ξ 210 1 S p 1
T 1 = 0 ξ 101 ξ 100 1 S e 1 T 1 = 0 ξ 201 ξ 200 S p 1
Total S e 2 1 S e 2 1Total 1 S p 2 S p 2 1
Observed Frequencies
CaseControl
T 2 = 1 T 2 = 0 Total T 2 = 1 T 2 = 0 Total
T 1 = 1 n 111 n 110 n 11 · T 1 = 1 n 211 n 210 n 21 ·
T 1 = 0 n 101 n 100 n 10 · T 1 = 0 n 201 n 200 n 20 ·
Total n 1 · 1 n 1 · 0 n 1 Total n 2 · 1 n 2 · 0 n 2
Table 2. Type I errors for P P V 1 = P P V 2 = 0.70 and N P V 1 = N P V 2 = 0.95 .
Table 2. Type I errors for P P V 1 = P P V 2 = 0.70 and N P V 1 = N P V 2 = 0.95 .
S e 1 = 0.5385   ,   S p 1 = 0.9744   ,   S e 2 = 0.5385   ,   S p 2 = 0.9744   ,   p = 10 %
ρ 1 = 0.25     ρ 2 = 0.25 ρ 1 = 0.50     ρ 2 = 0.50 ρ 1 = 0.75     ρ 2 = 0.75
n 1 n 2 Global α = 5 % Bonf.Global α = 5 % Bonf.Global α = 5 % Bonf.
50500.0310.0510.0290.0270.0480.0270.0040.0130.004
50750.0290.0590.0290.0250.0510.0260.0040.0170.005
501000.0280.0630.0300.0290.0610.0280.0080.0180.007
75750.0230.0610.0260.0310.0560.0280.0150.0340.017
1001000.0270.0630.0290.0230.0520.0240.0200.0430.019
2002000.0440.0860.0450.0320.0630.0310.0250.0500.026
5005000.0550.1070.0560.0580.1020.0570.0400.0770.039
S e 1 = 0.8615   ,   S p 1 = 0.8769   ,   S e 2 = 0.8615   ,   S p 2 = 0.8769   ,   p = 25 %
ρ 1 = 0.25     ρ 2 = 0.25 ρ 1 = 0.50     ρ 2 = 0.50 ρ 1 = 0.75     ρ 2 = 0.75
n 1 n 2 Global α = 5 % Bonf.Global α = 5 % Bonf.Global α = 5 % Bonf.
50500.0480.0940.0460.0180.0470.0180.0010.0070.002
50750.0530.1000.0510.0250.0630.0260.0020.0120.003
501000.0530.1060.0570.0340.0760.0320.0080.0230.008
75750.0590.1050.0550.0390.0870.0370.0070.0160.006
1001000.0590.1170.0590.0560.1020.0540.0110.0400.010
2002000.0580.0990.0570.0480.0940.0490.0440.0900.042
5005000.0520.0980.0530.0510.1010.0520.0490.0900.048
S e 1 = 0.9692   ,   S p 1 = 0.5846   ,   S e 2 = 0.9692   ,   S p 2 = 0.5846   ,   p = 50 %
ρ 1 = 0.25     ρ 2 = 0.25 ρ 1 = 0.50     ρ 2 = 0.50 ρ 1 = 0.75     ρ 2 = 0.75
n 1 n 2 Global α = 5 % Bonf.Global α = 5 % Bonf.Global α = 5 % Bonf.
50500.0260.0490.0260.0250.0610.0260.0060.0170.006
50750.0200.0490.0230.0190.0520.0240.0070.0280.010
501000.0190.0430.0230.0160.0450.0190.0100.0340.014
75750.0240.0650.0270.0200.0510.0270.0120.0380.017
1001000.0280.0660.0290.0210.0520.0250.0120.0420.019
2002000.0470.0880.0440.0340.0740.0320.0210.0580.026
5005000.0520.0990.0520.0500.0970.0490.0370.0770.034
Global: global hypothesis test based on the chi-square distribution. α = 5 % : individual hypothesis tests each one to an error of 5%. Bonf.: Bonferroni method.
Table 3. Type I errors for P P V 1 = P P V 2 = 0.85 and N P V 1 = N P V 2 = 0.95 .
Table 3. Type I errors for P P V 1 = P P V 2 = 0.85 and N P V 1 = N P V 2 = 0.95 .
S e 1 = 0.5312   ,   S p 1 = 0.9896   ,   S e 2 = 0.5312   ,   S p 2 = 0.9896   ,   p = 10 %   ,   0 ρ 1 1   ,   0 ρ 2 1
ρ 1 = 0.25     ρ 2 = 0.25 ρ 1 = 0.50     ρ 2 = 0.50 ρ 1 = 0.75     ρ 2 = 0.75
n 1 n 2 Global α = 5 % Bonf.Global α = 5 % Bonf.Global α = 5 % Bonf.
50500.0330.0560.0340.0200.0510.0240.0040.0140.004
50750.0240.0490.0240.0260.0500.0250.0050.0190.006
501000.0320.0570.0330.0300.0560.0300.0040.0160.004
75750.0340.0540.0330.0250.0520.0260.0140.0360.015
1001000.0270.0550.0260.0270.0550.0260.0170.0410.017
2002000.0330.0590.0310.0250.0500.0240.0220.0550.021
5005000.0460.0870.0490.0310.0680.0330.0180.0500.024
S e 1 = 0.85   ,   S p 1 = 0.95   ,   S e 2 = 0.85   ,   S p 2 = 0.95   ,   p = 25 %   ,   0 ρ 1 1   ,   0 ρ 2 1
ρ 1 = 0.25     ρ 2 = 0.25 ρ 1 = 0.50     ρ 2 = 0.50 ρ 1 = 0.75     ρ 2 = 0.75
n 1 n 2 Global α = 5 % Bonf.Global α = 5 % Bonf.Global α = 5 % Bonf.
50500.0230.0580.0220.0050.0300.0070.0010.0050.001
50750.0370.0770.0360.0140.0390.0150.0010.0080.001
501000.0490.0920.0480.0220.0560.0220.0010.0070.002
75750.0420.0870.0410.0250.0550.0250.0040.0140.004
1001000.0480.0950.0430.0280.0660.0270.0050.0250.005
2002000.0330.0590.0310.0250.0500.0240.0220.0550.021
5005000.0480.0970.0460.0560.1010.0510.0500.0990.049
S e 1 = 0.9562   ,   S p 1 = 0.8312   ,   S e 2 = 0.9562   ,   S p 2 = 0.8312   ,   p = 50 %   ,   0 ρ 1 1   ,   0 ρ 2 1
ρ 1 = 0.25     ρ 2 = 0.25 ρ 1 = 0.50     ρ 2 = 0.50 ρ 1 = 0.75     ρ 2 = 0.75
n 1 n 2 Global α = 5 % Bonf.Global α = 5 % Bonf.Global α = 5 % Bonf.
50500.0310.0720.0310.0140.0410.0150.0010.0070.001
50750.0320.0690.0330.0220.0490.0220.0050.0150.005
501000.0250.0570.0260.0250.0640.0260.0080.0250.008
75750.0380.0810.0370.0270.0540.0250.0060.0170.006
1001000.0390.0840.0380.0310.0730.0300.0080.0300.009
2002000.0330.0590.0310.0250.0500.0240.0220.0550.021
5005000.0510.0990.0490.0500.0970.0470.0430.0870.042
Global: global hypothesis test based on the chi-square distribution. α = 5 % : individual hypothesis tests, each one to an error of 5%. Bonf.: Bonferroni method.
Table 4. Powers for P P V 1 = 0.75 , N P V 1 = 0.95 , P P V 2 = 0.60 and N P V 2 = 0.95 .
Table 4. Powers for P P V 1 = 0.75 , N P V 1 = 0.95 , P P V 2 = 0.60 and N P V 2 = 0.95 .
S e 1 = 0.5357   ,   S p 1 = 0.9802   ,   S e 2 = 0.5455   ,   S p 2 = 0.9596   ,   p = 10 %   ,   0 ρ 1 0.9805   ,   0 ρ 2 0.6933
ρ 1 = 0.25     ρ 2 = 0.17 ρ 1 = 0.49     ρ 2 = 0.35 ρ 1 = 0.74     ρ 2 = 0.52
n 1 n 2 Global α = 5 % Bonf.Global α = 5 % Bonf.Global α = 5 % Bonf.
50500.0250.0560.0310.0230.0490.0240.0050.0190.007
50750.0370.0770.0360.0290.0630.0300.0100.0300.011
501000.0540.1030.0520.0420.0840.0380.0190.0460.016
75750.0380.0780.0380.0320.0660.0330.0180.0420.018
1001000.0530.0980.0470.0440.0810.0370.0310.0630.026
2002000.1990.2760.1800.2080.2860.1810.1680.2520.138
5005000.4950.5750.4620.5910.6680.5560.7200.7850.678
S e 1 = 0.8571   ,   S p 1 = 0.9048   ,   S e 2 = 0.8727   ,   S p 2 = 0.8061   ,   p = 25 %   ,   0 ρ 1 0.9354   ,   0 ρ 2 0.6614
ρ 1 = 0.23     ρ 2 = 0.17 ρ 1 = 0.47     ρ 2 = 0.33 ρ 1 = 0.70     ρ 2 = 0.50
n 1 n 2 Global α = 5 % Bonf.Global α = 5 % Bonf.Global α = 5 % Bonf.
50500.2590.3350.2300.2540.3450.2300.1950.3340.210
50750.4090.4960.3780.4540.5430.4240.4670.6060.470
501000.5050.5840.4620.5980.6770.5560.6830.7760.675
75750.4160.4980.3820.4690.5570.4360.5010.6080.476
1001000.5280.6060.4880.6250.6990.5790.7180.7930.685
2002000.8220.8620.7900.8910.9230.8730.9740.9830.964
5005000.9960.9990.996111111
S e 1 = 0.9643   ,   S p 1 = 0.6786   ,   S e 2 = 0.9818   ,   S p 2 = 0.3455   ,   p = 50 %   ,   0 ρ 1 0.7071   ,   0 ρ 2 0.50
ρ 1 = 0.18     ρ 2 = 0.13 ρ 1 = 0.35     ρ 2 = 0.25 ρ 1 = 0.53     ρ 2 = 0.38
n 1 n 2 Global α = 5 % Bonf.Global α = 5 % Bonf.Global α = 5 % Bonf.
50500.8900.9390.8930.9350.9690.9410.9770.9890.978
50750.9780.9900.9770.9950.9970.9930.9990.9990.999
501000.9950.9980.9950.9990.9980.999111
75750.9840.9920.9830.9950.9990.9940.99910.999
1001000.9980.9990.998110.999111
200200111111111
500500111111111
Global: global hypothesis test based on the chi-square distribution. α = 5 % : individual hypothesis tests, each one to an error of 5%. Bonf.: Bonferroni method.
Table 5. Powers for P P V 1 = 0.95 N P V 1 = 0.95 , P P V 2 = 0.75 and N P V 2 = 0.95 .
Table 5. Powers for P P V 1 = 0.95 N P V 1 = 0.95 , P P V 2 = 0.75 and N P V 2 = 0.95 .
S e 1 = 0.5278   ,   S p 1 = 0.9969   ,   S e 2 = 0.5357   ,   S p 2 = 0.9802   ,   p = 10 %   ,   0 ρ 1 0.9841   ,   0 ρ 2 0.3910
ρ 1 = 0.25     ρ 2 = 0.10 ρ 1 = 0.49     ρ 2 = 0.19 ρ 1 = 0.74     ρ 2 = 0.29
n 1 n 2 Global α = 5 % Bonf.Global α = 5 % Bonf.Global α = 5 % Bonf.
50500.0300.0590.0300.0190.0480.0200.0070.0190.008
50750.0310.0630.0320.0230.0540.0230.0090.0240.009
501000.0330.0640.0330.0300.0630.0300.0100.0310.009
75750.0330.0570.0320.0250.0550.0250.0150.0360.015
1001000.0340.0670.0330.0260.0590.0260.0260.0540.026
2002000.1230.1820.0940.1220.1770.0950.1080.1680.078
5005000.6660.7700.6620.6690.7810.6670.6990.8110.696
S e 1 = 0.8444   ,   S p 1 = 0.9852   ,   S e 2 = 0.8571   ,   S p 2 = 0.9048   ,   p = 25 %   ,   0 ρ 1 0.9511   ,   0 ρ 2 0.3779
ρ 1 = 0.24     ρ 2 = 0.09 ρ 1 = 0.48     ρ 2 = 0.19 ρ 1 = 0.71     ρ 2 = 0.28
n 1 n 2 Global α = 5 % Bonf.Global α = 5 % Bonf.Global α = 5 % Bonf.
50500.1720.2470.1420.1540.2340.1270.1070.1980.097
50750.4220.5370.3960.3980.5210.3780.3650.5410.367
501000.6270.7340.6150.6410.7550.6380.6740.7790.653
75750.4340.5490.4000.4320.5550.4100.4020.5520.391
1001000.6350.7530.6340.6550.7740.6560.6660.7960.683
2002000.9650.9810.9640.9770.9870.9740.9890.9940.988
500500111111111
S e 1 = 0.95   ,   S p 1 = 0.95   ,   S e 2 = 0.9643   ,   S p 2 = 0.6786   ,   p = 50 %   ,   0 ρ + 0.8388   ,   0 ρ 0.3333
ρ 1 = 0.21     ρ 2 = 0.08 ρ 1 = 0.42     ρ 2 = 0.17 ρ 1 = 0.63     ρ 2 = 0.25
n 1 n 2 Global α = 5 % Bonf.Global α = 5 % Bonf.Global α = 5 % Bonf.
50500.9290.9690.9420.9540.9830.9660.9650.9920.978
50750.9940.9980.9950.9990.9990.9990.99910.999
50100111111111
75750.9950.9980.9950.9970.9990.998111
100100111111111
200200111111111
500500111111111
Global: global hypothesis test based on the chi-square distribution. α = 5 % : individual hypothesis tests, each one to an error of 5%. Bonf.: Bonferroni method.
Table 6. Power of the test H 0 : P P V 1 = P P V 2 and type I error of the test H 0 : N P V 1 = N P V 2 when P P V 1 = 0.75 , N P V 1 = 0.95 , P P V 2 = 0.60 and N P V 2 = 0.95 .
Table 6. Power of the test H 0 : P P V 1 = P P V 2 and type I error of the test H 0 : N P V 1 = N P V 2 when P P V 1 = 0.75 , N P V 1 = 0.95 , P P V 2 = 0.60 and N P V 2 = 0.95 .
S e 1 = 0.5357   ,   S p 1 = 0.9802   ,   S e 2 = 0.5455   ,   S p 2 = 0.9596   ,   p = 10 %   ,   0 ρ 1 0.9805   ,   0 ρ 2 0.6933
ρ 1 = 0.25     ρ 2 = 0.17 ρ 1 = 0.49     ρ 2 = 0.35 ρ 1 = 0.74     ρ 2 = 0.52
n 1 n 2 PowerType I errorPowerType I errorPowerType I error
50500.0010.0300.0010.0240.0010.007
50750.0070.0290.0040.0270.0020.001
501000.0210.0290.0130.0270.0080.009
75750.0080.0310.0050.0300.0030.018
1001000.0260.0260.0170.0260.0100.021
2002000.1600.0240.1590.0280.1190.024
5005000.4490.0260.5440.0230.6720.020
S e 1 = 0.8571   ,   S p 1 = 0.9048   ,   S e 2 = 0.8727   ,   S p 2 = 0.8061   ,   p = 25 %   ,   0 ρ 1 0.9354   ,   0 ρ 2 0.6614
ρ 1 = 0.23     ρ 2 = 0.17 ρ 1 = 0.47     ρ 2 = 0.33 ρ 1 = 0.70     ρ 2 = 0.50
n 1 n 2 PowerType I errorPowerType I errorPowerType I error
50500.2110.0250.2230.0110.2100.002
50750.3460.0210.4200.0080.4700.002
501000.4480.0260.5510.0100.6750.001
75750.3520.0300.4350.0200.4790.005
1001000.4720.0270.5590.0250.6830.010
2002000.7850.0230.8700.0270.9610.019
5005000.9960.0240.9990.02610.027
S e 1 = 0.9643   ,   S p 1 = 0.6786   ,   S e 2 = 0.9818   ,   S p 2 = 0.3455   ,   p = 50 %   ,   0 ρ 1 0.7071   ,   0 ρ 2 0.50
ρ 1 = 0.18     ρ 2 = 0.13 ρ 1 = 0.35     ρ 2 = 0.25 ρ 1 = 0.53     ρ 2 = 0.38
n 1 n 2 PowerType I errorPowerType I errorPowerType I error
50500.8930.0030.9410.0020.9800.001
50750.9770.0030.9910.0020.9990.001
501000.9900.0020.9990.00210.001
75750.9830.0050.9930.00210.003
1001000.9970.00510.00410.002
20020010.01210.00610.007
50050010.02410.02410.019
Table 7. Power of the test H 0 : P P V 1 = P P V 2 and type I error of the test H 0 : N P V 1 = N P V 2 P P V 1 = 0.95 , N P V 1 = 0.95 , P P V 2 = 0.75 and N P V 2 = 0.95 .
Table 7. Power of the test H 0 : P P V 1 = P P V 2 and type I error of the test H 0 : N P V 1 = N P V 2 P P V 1 = 0.95 , N P V 1 = 0.95 , P P V 2 = 0.75 and N P V 2 = 0.95 .
S e 1 = 0.5278   ,   S p 1 = 0.9969   ,   S e 2 = 0.5357   ,   S p 2 = 0.9802   ,   p = 10 %   ,   0 ρ 1 0.9841   ,   0 ρ 2 0.3910
ρ 1 = 0.25     ρ 2 = 0.10 ρ 1 = 0.49     ρ 2 = 0.35 ρ 1 = 0.74     ρ 2 = 0.29
n 1 n 2 PowerType I errorPowerType I errorPowerType I error
50500.0010.0300.0010.02400.006
50750.0020.0320.0020.0270.0010.008
501000.0050.0310.0030.0270.0020.008
75750.0020.0320.0010.0260.0010.017
1001000.0100.0300.0060.0220.0050.023
2002000.0710.0270.0680.0280.0560.022
5005000.6540.0250.6780.0280.6930.025
S e 1 = 0.8444   ,   S p 1 = 0.9852   ,   S e 2 = 0.8571   ,   S p 2 = 0.9048   ,   p = 25 %   ,   0 ρ 1 0.9511   ,   0 ρ 2 0.3779
ρ 1 = 0.24     ρ 2 = 0.09 ρ 1 = 0.48     ρ 2 = 0.19 ρ 1 = 0.71     ρ 2 = 0.28
n 1 n 2 PowerType I errorPowerType I errorPowerType I error
50500.1200.0250.1180.0120.0970.001
50750.3780.0280.3710.0120.3760.001
501000.6040.0260.6340.0120.6520.001
75750.3820.0270.3960.0220.3880.005
1001000.6220.0310.6440.0260.6790.012
2002000.9630.0250.9740.0240.9870.026
50050010.02810.02410.024
S e 1 = 0.95   ,   S p 1 = 0.95   ,   S e 2 = 0.9643   ,   S p 2 = 0.6786   ,   p = 50 %   ,   0 ρ 1 0.8388   ,   0 ρ 2 0.3333
ρ 1 = 0.21     ρ 2 = 0.08 ρ 1 = 0.42     ρ 2 = 0.17 ρ 1 = 0.63     ρ 2 = 0.25
n 1 n 2 PowerType I errorPowerType I errorPowerType I error
50500.9420.0020.9650.0010.9780
50750.9950.0030.99900.9970
5010010.00210.00210
75750.9960.0060.9970.0030.9990
10010010.01010.00610.002
20020010.02910.01510.010
50050010.02610.02410.024
Table 8. Effect of a misspecification of the prevalence.
Table 8. Effect of a misspecification of the prevalence.
Type I errors
P P V 1 = P P V 2 = 0.90   ,   N P V 1 = N P V 2 = 0.80 S e 1 = 0.2571   ,   S p 1 = 0.9905   ,   S e 2 = 0.2571   ,   S p 2 = 0.9905   ,   p = 25 %   ,   ρ 1 = 0.75   ,   ρ 2 = 0.75
p = p = 25 % p = 20 % p = 22.50 % p = 27.50 % p = 30 %
n 1 n 2 GlobalBonf.GlobalBonf.GlobalBonf.GlobalBonf.GlobalBonf.
50500.0010.0010.0010.0020.0010.0020.0010.0010.0010.001
50750.0010.0020.0010.0020.0010.0020.0010.0020.0010.002
501000.0020.0030.0020.0030.0020.0030.0020.0030.0020.003
75750.0030.0070.0030.0080.0030.0080.0030.0070.0030.007
1001000.0060.0100.0060.0110.0060.0110.0060.0090.0060.009
2002000.0100.0200.0100.0200.0100.0200.0100.0190.0100.019
5005000.0200.0240.0200.0240.0200.0240.0200.0230.0200.023
Powers
P P V 1 = 0.95   ,   P P V 2 = 0.75   ,   N P V 1 = 0.95   ,   N P V 2 = 0.95 S e 1 = 0.8444   ,   S p 1 = 0.9852   ,   S e 2 = 0.8571   ,   S p 2 = 0.9048   ,   p = 25 %   ,   ρ 1 = 0.71   ,   ρ 2 = 0.28
p = p = 25 % p = 20 % p = 22.50 % p = 27.50 % p = 30 %
n 1 n 2 GlobalBonf.GlobalBonf.GlobalBonf.GlobalBonf.GlobalBonf.
50500.1720.1420.1450.1390.1430.1350.1420.1210.1390.117
50750.4220.3960.3810.3820.3850.3820.3740.3800.3750.379
501000.6470.6150.6000.5870.6270.6040.6160.6140.5980.596
75750.4340.4000.4140.3960.4150.3960.3970.3940.3900.390
1001000.6350.6340.6180.6040.6320.6120.6390.6200.6230.615
2002000.9650.9640.9410.9310.9510.9520.9580.9560.9420.933
5005001111111111
RRMSEs of the estimators of PVs of Test 1
P P V 1 = P P V 2 = 0.90   ,   N P V 1 = N P V 2 = 0.80 S e 1 = 0.2571   ,   S p 1 = 0.9905   ,   S e 2 = 0.2571   ,   S p 2 = 0.9905   ,   p = 25 %   ,   ρ 1 = 0.75   ,   ρ 2 = 0.75
p = p = 25 % p = 20 % p = 22.50 % p = 27.50 % p = 30 %
n 1 n 2 P P V ^ i N P V ^ i P P V ^ i N P V ^ i P P V ^ i N P V ^ i P P V ^ i N P V ^ i P P V ^ i N P V ^ i
505027.41.836.65.531.82.629.33.834.56.4
507520.51.727.05.123.52.821.93.625.56.1
5010016.11.721.95.218.82.917.83.521.16.0
757520.11.426.55.123.12.621.43.524.16.1
10010015.41.219.24.918.12.516.13.318.65.9
2002008.00.911.74.79.92.48.73.010.25.6
5005004.40.57.14.15.42.24.92.85.75.5
Global: global hypothesis test based on the chi-square distribution. Bonf.: Bonferroni method.
Table 9. Diagnosis of coronary heart disease.
Table 9. Diagnosis of coronary heart disease.
Observed Frequencies
CaseControl
T 2 = 1 T 2 = 0 Total T 2 = 1 T 2 = 0 Total
T 1 = 1 771087 T 1 = 1 426
T 1 = 0 61218 T 1 = 0 13101114
Total8322105Total17103120
Table 10. Type I errors and powers when 0.5 is added to the samples in which n i 10 = n i 01 = 0 .
Table 10. Type I errors and powers when 0.5 is added to the samples in which n i 10 = n i 01 = 0 .
Type I Errors
P P V 1 = P P V 2 = 0.70   ,   N P V 1 = N P V 2 = 0.95 S e 1 = 0.5385   ,   S p 1 = 0.9744   ,   S e 2 = 0.5385   ,   S p 2 = 0.9744   ,   p = 10 %
ρ 1 = 0.25 ρ 2 = 0.25 ρ 1 = 0.50 ρ 2 = 0.50 ρ 1 = 0.75 ρ 2 = 0.75
n 1 n 2 P ¯ 1 P ¯ 2 Global α = 5 % Bonf.Global α = 5 % Bonf.Global α = 5 % Bonf.
50503.378.10.0280.0650.0300.0230.0560.0250.0120.0400.015
50753.163.70.0380.0750.0390.0260.0600.0260.0120.0410.014
501002.751.90.0470.0960.0460.0350.0740.0370.0110.0410.015
75750.464.30.0370.0740.0370.0260.0620.0270.0150.0450.019
1001000.151.80.0400.0850.0380.0320.0900.0340.0210.0480.023
200200022.70.0610.1040.0600.0450.0880.0420.0240.0550.025
50050002.80.0520.0990.05240.0560.1010.0480.0440.0940.045
Powers
P P V 1 = 0.75   ,   P P V 2 = 0.60   ,   N P V 1 = 0.95   ,   N P V 2 = 0.95 S e 1 = 0.8571   ,   S p 1 = 0.9048   ,   S e 2 = 0.8727   ,   S p 2 = 0.8061   ,   p = 25 %   ,   0 ρ 1 0.9354   ,   0 ρ 2 0.6614
ρ 1 = 0.25 ρ 2 = 0.25 ρ 1 = 0.50 ρ 2 = 0.50 ρ 1 = 0.75 ρ 2 = 0.75
n 1 n 2 P ¯ 1 P ¯ 2 Global α = 5 % Bonf.Global α = 5 % Bonf.Global α = 5 % Bonf.
505014.818.10.2970.3690.2690.3420.4270.3060.3690.4840.340
507514.59.60.4170.5010.3790.4920.5760.4490.6040.6880.568
5010014.25.50.5150.5900.4760.6290.6930.5760.7390.7980.704
75755.89.70.4240.5060.3920.5020.5840.4560.6220.7050.579
1001002.55.50.5250.6030.4860.6240.6930.5830.7610.8190.728
2002000.10.70.8170.8630.7860.9100.9300.8760.9750.9830.965
500500000.9960.9990.997111111
P ¯ 1 : average proportion (in %) of case samples in which 0.5 has been added. P ¯ 2 : average proportion (in %) of control samples in which 0.5 has been added. Global: global hypothesis test based on the chi-square distribution. α = 5 % : individual hypothesis tests, each one to an error of 5%. Bonf.: Bonferroni method.
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Regad, S.B.; Roldán-Nofuentes, J.A. Global Hypothesis Test to Compare the Predictive Values of Diagnostic Tests Subject to a Case-Control Design. Mathematics 2021, 9, 658. https://doi.org/10.3390/math9060658

AMA Style

Regad SB, Roldán-Nofuentes JA. Global Hypothesis Test to Compare the Predictive Values of Diagnostic Tests Subject to a Case-Control Design. Mathematics. 2021; 9(6):658. https://doi.org/10.3390/math9060658

Chicago/Turabian Style

Regad, Saad Bouh, and José Antonio Roldán-Nofuentes. 2021. "Global Hypothesis Test to Compare the Predictive Values of Diagnostic Tests Subject to a Case-Control Design" Mathematics 9, no. 6: 658. https://doi.org/10.3390/math9060658

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop