1 Introduction

Alzheimer’s disease (AD) is a progressive neurodegenerative disorder characterized by a gradual loss of cognitive function and ability to perform activities of daily living (ADL) [1, 2]. Behavioral and psychological symptoms can emerge, including physical aggression, restlessness, inappropriate social behaviors, and agitation, with behavioral symptoms worsening as the disease progresses [3]. Cognition, function and behavioral domains can all be assessed individually, with the Committee for Medicinal Products for Human Use (CHMP) also recommending that symptomatic improvements be assessed using a global assessment of response [4].

The burden of AD is expected to increase with the aging population, with an anticipated 115 million people globally living with dementia by 2050 [5]. AD is associated with a substantial economic burden. The total annual cost of dementia across Europe in 2010 was estimated to be €105.2 billion, accounting for 13 % of the costs for all brain disorders [6].

To date, there is no cure available for AD and prevention of further worsening of symptoms represents the most realistic treatment goal [7, 8]. Memantine is a moderate-affinity, noncompetitive, voltage-dependent N-methyl-d-aspartate receptor antagonist with fast on–off kinetics [9]. It is licensed for the treatment of moderate to severe AD, represented by patients with a mini mental state examination (MMSE) score of <20 [10]. Memantine can be used in treatment-naïve patients and patients withdrawn from acetylcholinesterase inhibitors (AChEIs), or as an add-on treatment in patients already stabilized on an AChEI, most commonly donepezil.

The efficacy of memantine has been evaluated in several clinical trials, with memantine assessed both as a monotherapy [11–14] and as an add-on therapy in patients already receiving AChEIs [15, 16]. Improvements in AD domains versus placebo showed considerable variation in whether these outcomes were significantly improved. Meta-analysis of the data from several clinical trials is needed to understand the efficacy of memantine based on a pooled analysis. However, as a result of the heterogeneity of the trial populations, synthesis of the meta-analysis data can be challenging with numerous methodological decisions required.

Several meta-analyses have been published that aimed to provide conclusions on the efficacy of memantine in the treatment of AD from clinical trials. The objective of this article is to review the methodologies of these analyses, to assess the impact of different approaches on the pooled results, and to generate general conclusions on the efficacy of memantine in AD.

2 Published Meta-Analyses of Memantine Data

Five meta-analyses of memantine data are considered. This selection does not stem from a systematic literature review but was aimed at covering a wide range of approaches (for the selection of studies, patient populations, analysis strategy, and analysis method) and authors, but also based on a similar availability of evidence. In 2006, the Cochrane Collaboration published a meta-analysis of memantine in AD, vascular dementia, and mixed dementia [1]. A meta-analysis published in 2007 by Winblad was conducted during the European regulatory process when the memantine license was extended from moderately severe to severe patients, to also include patients with moderate AD [2]. The meta-analysis published in 2007 by Doody considered the overall efficacy and safety of memantine across the spectrum of AD severity [17]. Meta-analyses have also been conducted as part of national health technology assessments. In 2009, the German Institute for Quality and Efficiency in Health Care (IQWiG) published their evaluation report on memantine in AD [18]. In 2010, as part of the National Institute for Health and Clinical Excellence (NICE) review of AD therapies, the independent Peninsula Technology Assessment Group (PenTAG) conducted a systematic review and meta-analysis [19].

All five meta-analyses incorporated data from the clinical development program for memantine in AD. This comprised six randomized, double-blind, placebo-controlled trials of 6 months duration, designed in line with current recommendations of the European Medicines Agency (EMA). The trials assessed the efficacy of memantine across key AD domains (cognition, ADL, global assessment, and behavior) using standard scales. The methodologies of the trials were similar but with important differences in the included patient populations, both in terms of AD severity and background treatment with AChEIs (Table 1). Another important point is that in distinct populations of varying severities, different scales are sometimes used to measure the same concept. For example, both the Alzheimer’s disease assessment scale-cognitive subscale (ADAS-cog) and severe impairment battery (SIB) are used to assess cognition.

Table 1 Overview of the design of memantine clinical trials
Full size table

While these six pivotal registration studies were the main data sources, not all meta-analyses included all trials. Furthermore, one additional trial conducted more recently was included in the meta-analysis conducted by IQWiG: Lu-10116, the pivotal trial for the authorization of memantine in China, assessed memantine as a monotherapy in Chinese patients with moderate to severe AD over a 4-month period [20].

2.1 Cochrane

This meta-analysis included the six pivotal trials and utilized only data available in the primary publications with no posthoc analyses [1]. Analyses were conducted separately for trials in patients with moderately severe to severe AD, and with mild to moderate AD. The meta-analysis does not present a summary that encompasses the total licensed population for memantine, and the second analysis includes off-label use.

2.2 Winblad

This meta-analysis included all six pivotal clinical trials [2]. In accordance with the indication for memantine, only moderate AD patients from the clinical trials that included mild to moderate AD patients were included in the meta-analyses, and patients with mild AD were excluded. As part of the evidence dossier prepared by NICE, the manufacturer of memantine (Lundbeck) undertook an interaction analysis using the same set of studies and the same analysis population as in the Winblad meta-analysis to ascertain the impact of baseline disease severity and prior or concomitant AChEI use [21].

2.3 Doody

As with the Cochrane and Winblad publications, this included the six pivotal clinical trials for memantine [17]. The entire population from these trials were included, with mild AD representing off-label use considered.

2.4 IQWiG

The IQWiG meta-analysis included the six pivotal trials plus clinical trial Lu-10116 [20]. In line with the memantine indication, mild patients were excluded. In the meta-analysis, severe patients from study MEM-MD-02 [15] were excluded because donepezil is not indicated in this group.

2.5 PenTAG

This analysis considered trials for memantine as monotherapy and as combination therapy separately [19]. In monotherapy, studies MRZ-9605 [11] and MEM-MD-01 [12] were included. Trials Lu-99679 [13] and MEM-MD-10 [14] were excluded on the basis that they included mild off-label use patients. Furthermore, posthoc analyses of moderate AD patients from these trials were not considered by PenTAG to be reported in sufficient detail to allow their inclusion in meta-analyses. In the combination analysis, trials MEM-MD-02 [15] and MEM-MD-12 [16] were considered. Study MEM-MD-12 included patients with mild to moderate disease. There was therefore a lack of consistency between the analysis of the monotherapy and combination trials, with the exclusion of trials in patients with mild to moderate AD for monotherapy yet the inclusion of these trials in the combination analysis. This inconsistency was highlighted by Lundbeck during the NICE review process [21]. The inclusion of study MEM-MD-12 in the combination analysis was justified by PenTAG based on the MMSE score upper range at baseline being 20.37 [22]. PenTAG stated that as this value was only minimally over 20 (the threshold for moderate disease) the study could be included. However, this justification does not explain the discrepancy in approach between the monotherapy and combination analyses. Figure 1 provides the MMSE scores at baseline in the MEM-MD-10 and MEM-MD-12 studies. The cutoff used by PenTAG to exclude MEM-MD-10 was at least 20 % of patients with mild disease. However, the patients included in the two studies had very similar disease severities at baseline with both above the 20 % threshold: 32.5 % of patients in MEM-MD-10 had mild AD at baseline compared to 30.4 % in MEM-MD-12. Although the concerns regarding inconsistency were raised, these were not amended in the final PenTAG report or the NICE final appraisal determination [23]. However, for the purpose of this analysis and to ensure consistency of approach, the combination results will be based on study MEM-MD-02 only, in line with the approach adopted for the monotherapy evaluation.

Fig. 1
figure 1

Baseline severity scores (mini mental state examination; MMSE) in studies MEM-MD-10 and MEM-MD-12

Full size image

3 Methodological Issues

Due to the heterogeneous nature of patients in memantine trials, the synthesis of evidence can be approached in several ways with a number of key points to consider when determining the most appropriate analysis. The first decision relates to the severity of included patients. There are no studies for memantine that consider the complete licensed indication only. A selection of trials in the moderately severe to severe patient population only excludes an assessment of memantine in moderate AD patients above the moderately severe threshold. The limitation of this approach is that a proportion of the population corresponding to the memantine indication are not considered. For moderate patients to be considered, the trials in mild to moderate patients must be included in the data synthesis. If these trials are included in their entirety, the meta-analysis will consider the use of memantine in mild AD, which represents off-label use and may make the analysis less clinically relevant. However, subgroup analysis of the moderate patients only from the mild to moderate trials represents a posthoc review of data that could potentially break trial randomization, which was not stratified on baseline severity. The Cochrane Collaboration recommend that subgroup analyses in meta-analyses should be kept to a minimum and only conducted when there is a clinical rationale [24].

Another methodological decision is the analysis of data according to the presence or absence of treatment with an AChEI. Combining data across these populations gives a greater power to the analysis, but also results in mixing of distinct and heterogeneous patient groups.

The approach to accounting for missing data must also be considered. Analysis can be conducted on observed cases (OC), last observation carried forward (LOCF), or as reported in the original publications. In general, the LOCF method is preferred because it is closer to the intent-to-treat principle and better preserves the original study sample size. However, in chronic and progressive conditions such as AD, the LOCF method can, in some cases, artificially overestimate the efficacy of treatment [25]. For example, if a patient withdraws from the study, due to the degenerative nature of AD the LOCF analysis will overestimate the effect by simulating stability when deterioration is more likely. Whether the LOCF analysis will overestimate or underestimate the treatment effect depends on the balance, timing, and reason of withdrawals between the active and control groups. In this case, the pooled withdrawal rate across the six pivotal trials was lower with memantine than placebo [2].

The methodological approach of each meta-analysis is summarized in Table 2. In a meta-analysis, data can be presented as a mean difference (MD; sometimes referred to as weighted mean differences) or a standardized mean difference (SMD). A MD is used when all data being pooled has been assessed with the same scales and this represents an absolute effect size. This is generally the preferred approach because the results are expressed in terms of the original outcome measures making them easy to interpret. However, in cases when a single outcome has been assessed with different measures, the SMD must be used. This is relevant to AD because there are a number of different instruments that can be used to measure the same domain. For example, the ADAS-cog and SIB are used to assess cognition in mild to moderate AD patients and in moderately severe to severe AD patients, respectively [26].

Table 2 Summary of methodology used in the meta-analyses
Full size table

An examination of the methodological choices of each meta-analysis was conducted and the results compared. The heterogeneity of the results was examined using the I 2 measure, which describes the percentage of total variation across studies that is due to heterogeneity and not chance [27]. An I 2 value of 0 % indicates no heterogeneity while 25, 50, and 75 % are the thresholds for low, moderate, and high heterogeneity, respectively.

4 Reports of Efficacy from Different Meta-Analyses

The results from the individual meta-analyses are provided in Table 3, with significant results for memantine compared to placebo highlighted. Heterogeneity is summarized in Table 4.

Table 3 Results of meta-analyses—difference versus placebo
Full size table
Table 4 Summary of heterogeneity assessments in meta-analyses
Full size table

For cognition, the impact of memantine was significant versus placebo in the majority of meta-analyses. The effect size of memantine was greater in the moderately severe to severe population than in the mild to moderate population, as highlighted by the Cochrane analysis [1]. In the two meta-analyses that considered the licensed population for memantine (Winblad [2] and IQWiG [18]), the results for cognition were similar. The slight variation can be explained by the inclusion in the latter of study Lu-10116 and different approaches to selecting data. For cognition, the grouping or not of trials based on background treatment had an impact on results. The PenTAG analysis for monotherapy in moderately severe to severe patients (MRZ-9605 [11] and MEM-MD-01 [12]) was the only review for which cognition was not significantly improved with memantine. There was a substantial difference in effect between study MEM-MD-01, with no significant impact of memantine on cognition at week 24, and MRZ-9605, where results were significant.

When considering the effect of memantine on ADL, the selection of patients according to severity has some impact on results. The effect of memantine was consistently significant for ADL in moderately severe to severe patients and was not significant in mild to moderate patients. The efficacy was significant when the total spectrum of AD severity was considered (Doody analysis).

The results for memantine on behavioral disturbances were disparate. There was a significant benefit in moderately severe to severe patients when both monotherapy and combination trials were considered. In this patient population, the efficacy was significant when only combination trial MEM-MD-02 was considered, and was not significant in monotherapy studies. The impact on behavior was also not significant when the entire patient spectrum was considered, as reported in the Doody analysis. For the licensed indication, a significant efficacy was reported in the Winblad analysis, although the authors highlight that interpretation of the results is limited because the LOCF analysis demonstrates considerable heterogeneity. This was the only analysis in the Winblad article that demonstrated heterogeneity. It is of note that the analysis most similar to Winblad conducted by IQWiG did not report a significant benefit for behavior. Discrepancies in the behavior outcome results do not appear to be easily explained by methodological differences.

For overall assessment of disease, memantine efficacy was significant across all meta-analyses and the effect was generally homogeneous, with the exception of the Cochrane analysis in mild to moderate patients. In line with the results on cognition, the effect with memantine was greater in the more severe patient population. In the PenTAG analysis, the efficacy for overall assessment was greater for memantine as a monotherapy than in combination therapy, which was based on study MEM-MD-02 only in this case. For the licensed indication, the effect size of memantine was reported to be similar in the Winblad and IQWiG meta-analyses, and again small differences can be explained by an additional trial included in the latter and different approaches to data selection.

5 Factors Contributing to Different Results

Patients in the pivotal memantine studies were very heterogeneous and meta-analyses can therefore be conducted in a number of ways. Interpretation of the data must be considered within the context of the methodological approach. All approaches are valid and each methodological decision is associated with strengths and limitations. For example, exclusion of mild patients no longer ensures trial randomization. However, the other approach is to either include all patients from mild to moderate trials, giving an assessment of memantine outside the licensed indication, or to exclude these trials and therefore moderate patients.

5.1 Disease Severity

We have considered the impact of the selected patients, in terms of severity and concomitant treatment, on the meta-analysis results. While patient selection can go some way to explaining differences in outcomes, for example the effect of memantine on function in mild to moderate versus moderate to severe patients, these factors do not explain all observed variations. This is highlighted by the interaction analyses on the Winblad meta-analysis; in moderate to severe patients there was no significant interaction between treatment effect and the baseline factors of disease severity or use of AChEIs. This supports the strategy of combining data from the six clinical trials.

5.2 Weighting of Negative Trials

Possible explanations for differences in results could be related to other factors, such as insufficiently powered subgroup analyses or the relative weight of negative studies. For example, the PenTAG monotherapy analysis reported no significant results for memantine in cognition in moderately severe to severe patients. When compared to the Cochrane review, which included the same severity population but included combination study MEM-MD-02, the results were significant. This can be explained by the different weights in each analysis given to study MEM-MD-01 for which no significant benefit for memantine was reported. The challenge in understanding the differences in outcomes between memantine studies results from there being no active treatment comparator arm. It is therefore impossible to ascertain whether nonsignificant results are a result of negative trials, where the nonsignificant difference between active treatment and placebo is a consequence of the treatment itself, or a failed trial where the inability to show significance is due to flaws in the trial design and/or execution.

5.3 Selection of Trials

The selection of trials is also key. Given the disparity in results between individual studies for memantine, when a lower number of trials are included in the meta-analysis, the impact of extreme results in a given study on overall heterogeneity is higher. Broader inclusion criteria are more likely to give a homogenous result reflective of the true effect of memantine. For moderate to severe patients, this is highlighted in the Winblad and IQWiG analyses, which included all trials across the licensed indication and were associated with low heterogeneity.

5.4 Baseline Behavioral Disorders

It is also worth noting that the efficacy of memantine has been reported in patients with agitation, aggression, or psychosis at baseline [28]. In fact, the efficacy in this population was greater than that reported in patients without these symptoms at baseline across all domains and this may further explain differences between analyses dependent on the proportion of patients with these behavioral disturbances at baseline [29].

Of the domains reviewed, the results for behavioral disorders are the most disparate and it is challenging to understand what drives these differences. Generally, in patients with behavioral disturbances, neuropsychiatric symptoms are chronically present. However, individual symptoms often show an intermittent course and patients may periodically experience different symptom severities [30, 31]. This makes quantifying behavioral symptoms and assessing their change over time difficult. The scale used to assess the behavioral disorders, the neuropsychiatric inventory (NPI) [32, 33], considers 12 types of different and sometimes antagonistic neuropsychiatric behaviors. Each behavior is scored in terms of frequency and severity, and these are multiplied to give a total score. The calculation of NPI total score is such that two patients with very different profiles in terms of behavioral symptoms could have similar NPI scores. The heterogeneity of AD and the scoring system of the NPI means that this can be difficult to interpret, and may explain the disparate meta-analysis results. It is also worth noting that none of the memantine studies included NPI as a primary efficacy endpoint, potentially leading to problems of powering. Two studies (MEM-MD-10 and MEM-MD-02) did demonstrate a significant effect of memantine compared to placebo for NPI [14, 15].

Given the heterogeneity of behavioral symptoms in AD, a global score may not be the best approach. A more appropriate method could be to analyze different types of behavioral problems separately. This was performed in a pooled analysis of individual patient data that considered changes from baseline in both total NPI score and in single NPI items [34], in moderate to severe AD patients treated with memantine in the six pivotal trials. At 24 to 28 weeks, memantine was reported to be associated with significant improvements versus placebo in both total NPI score (p = 0.008) and the individual symptoms of delusions (p = 0.001), agitation/aggression (p = 0.001), and irritability/lability (p = 0.005). At this time point, memantine was not associated with a significant impact on the other nine individual NPI single items. This highlights that when the NPI total score is analyzed, there is substantial data contributing to the score that is not significant and contributes “noise” to the system. The improvements observed with memantine in agitation and aggression were particularly significant, because these are not only among the most common behavioral symptoms of AD, but are most commonly associated with emotional strain for caregivers, rapid disease progression, and early institutionalization [34, 35]. Another important analysis relates to the emergence of symptoms in subsets of patients asymptomatic for the individual NPI items at baseline. At week 24/28, significantly more memantine-treated patients remained asymptomatic in terms of agitation/aggression, irritability/lability, and night-time behavior compared to placebo.

5.5 Additional Trials

In addition to clinical trials presented here that have been included in meta-analyses, more recent clinical trials have been completed: Asubio IE-2101 [36], Forest MEM-MD-22 [37], Lundbeck 10112 [38], and Lundbeck 10158 [39]. These included both positive and nonconclusive studies. An update of the Winblad meta-analysis including three of these additional clinical trials (Asubio IE-2101, Forest MEM-MD-22, Lundbeck 10112) has been performed [29]. Conclusions were that inclusion of the additional trials had no influence on the results and all conclusions remained unchanged.

Across all domains, the efficacy of memantine compared to placebo has been demonstrated in individual clinical trials. This highlights that the benefits reported in pooled analyses represent a real effect of memantine therapy and are not an artefact of pooled data.

6 Conclusions

The efficacy of memantine was consistently demonstrated versus placebo across the cognition, ADL, and overall assessment domains within the licensed indication. Benefit of memantine on behavioral symptoms was also demonstrated in several meta-analyses, although a simple analysis of the total NPI score might not be sufficient to have a comprehensive view on the nature of this benefit. The meta-analyses of memantine reported here highlight that the methodological approach has an impact on the results, but differences between meta-analyses cannot be fully explained by the selection of the trials and analysis method. Overall, individual clinical trial results and the consistency of meta-analyses results demonstrate that memantine represents a valuable treatment option in AD.