We thank A/Prof. Hurley for drawing further attention to the need for any meta-analysis to examine its candidate constituent trials for factors that might introduce bias. In the quoted example, the composite trials were potentially flawed in that antibiotic prophylaxis for ventilator-associated pneumonia might have increased the rate of pneumonia in control groups rather than decreasing the rate in intervention groups. We agree that identifying this situation is impossible if the data from the constituent trials are simply subjected to mathematical aggregation in a meta-analysis, whether the analysis is of aggregate or individual patient data (IPD). We therefore further agree that any meta-analysis must investigate the validity of its constituent studies. However, we disagree that coming to a misleading conclusion is more likely when individual patient data are properly used. Individual patient data increase the power of a meta-analysis to detect both overall and subgroup effects. As when using any more powerful tool, more care must be exercised to achieve the greater desired effects without undue danger.

From a statistical perspective, an IPD meta-analysis is equivalent to the analysis required for a multi-centre study, where the increased number of patients and centres increases both the power of the study and the generalisability of the findings [1, 2]. In accordance with the appropriate analysis techniques for multicentre studies [3], we agree with Hurley that it is imperative for any meta-analysis to examine heterogeneity between studies. Where heterogeneity is not too great and treatment numbers are balanced within studies, it is possible to control for heterogeneity using multivariate techniques. Where true heterogeneity is found to exist, the interpretation of the main treatment effect will be controversial. However, we must disagree with the suggestion that an IPD meta-analysis per se would be more likely to produce an incorrect conclusion than one based on aggregate data. Any such effect would occur only if data had been incorrectly analysed without due care.

Not only does individual patient data provide a vehicle to more accurately assess heterogeneity, it facilitates the use of multivariate analysis to evaluate treatment-by-study interactions and further enables adjustment of the individual studies to more accurately define the true treatment effect. In the example quoted by Hurley, analysis of individual patient data—assuming the required covariates are available—is really the only way of testing the hypothesis advanced. For example, were the infections in the trials’ control groups atypical (involving perhaps atypical organisms) compared to those in observational studies? Contrary to the argument advanced, there could be no more accurate way to retrospectively analyse studies than to use individual patient data. It is unfortunate that IPD meta-analysis carries the term “meta-analysis” in the title, because the well documented flaws associated with aggregate meta-analysis [4] tend by default also to be associated with IPD meta-analysis, when this is actually not the case. Statistically, an IPD-MA is more like a multi-centre study, and should be viewed accordingly.