Alzheimer disease currently affects more than 35 million people worldwide. After initial enthusiasm for the acetylcholinesterase inhibitors and memantine, their symptomatic efficacy and long-term value have been questioned by public health authorities. Against this backdrop of controversy, we have used the largest clinical trial registry to analyze clinical research on these drugs. We focused on the clinical relevance of the trials and on their publication rate.
We used the ClinicalTrials.gov website, which is an open registry of clinical trials. We conducted our literature search in January 2015 on clinical intervention and extracted data for the period from 2004 to 2011 inclusive, i.e. 8 full years. We search publication on ClinicalTrials.gov, Medline and Google Scholar.
The literature search found 54 studies published of 107 studies that met the selection criteria. About the published trials, the mean duration was 7.5 months, the mean age of participants was between 70 and 79 in 78% of the trials. In terms of results, 25 trials were positive, 25 doubtful, and 4 negative. Depending on the trial methodology, 19 of 28 placebo-controlled trials were classified as doubtful or negative. Of the 28 placebo-controlled trials, only 6 lasted one year or more, with 2 negative results and 4 doubtful results. Only half of the studies were published, and those describing a pharmacological intervention versus placebo were most often neither positive nor clinically relevant.
Only half of the studies were published, and those describing a pharmacological intervention versus placebo were most often neither positive nor clinically relevant. The pharmaceutical industry does not seem to attempt to answer the questions of the health authorities concerning the long-term effects and the clinical relevance of these drugs.
Alzheimer disease currently affects more than 35 million people worldwide, mainly the over-80s [1]. It progresses inexorably to dementia, which causes severe functional impairment. The search for an effective treatment is a public health priority. Only symptomatic treatments are available [1] and involve the use of two classes of drugs: acetylcholinesterase inhibitors (donepezil, galantamine, and rivastigmine) and antagonists of certain glutamatergic receptors (memantine) [2]. The first acetylcholinesterase inhibitor, tacrine, has had marketing authorization in 1994. But, due to hepatics adverse events, this molecule was withdrawn of market in 2004. More recently, monoclonal antibodies directed to the components of amyloid plaque (bapineuzumab for N-terminal part of Abêta, and solanezumab for soluble Abêta) have been developed. Unfortunately, the publication results have been negative [3,4]. After initial enthusiasm for the three acetylcholinesterase inhibitors and memantine, which were developed in the 1990s, over the last decade or so their symptomatic efficacy and long-term value have been questioned by public health authorities, notably in Great Britain [5] and France [6]. At best their efficacy is deemed modest and limited to the very short term. There is no high level of proof that these drugs change the disease trajectory or have an effect on loss of functional autonomy or mortality. Their clinical relevance has therefore come under scrutiny. What is more, serious side effects have been reported and are more frequent and severe in older patients and in those taking multiple medications [7]. Lastly, as in other medical fields, there is the question of the availability of clinical research findings on these drugs and possible overestimation of their efficacy because of failure to report negative studies [8].
Against this backdrop of controversy, we have used the largest clinical trial registry to analyze clinical research on these drugs conducted over the last ten years. We focused on the clinical relevance of the trials and on their publication rate, echoing the concerns of the health authorities.
The United States National Library of Medicine at the National Institutes of Health manages the ClinicalTrials.gov website, which is an open registry of clinical trials. This registry is the world's largest database of publicly and privately supported clinical studies [8-10]. ClinicalTrials.gov allows free access to information essential to understanding the research conducted (sponsor, country, primary and secondary objectives, inclusion and exclusion criteria, study population size, participating centers...). It has a research engine that expedites analysis of information concerning the clinical trials. Multicenter trials with the same protocol are considered as a single study in the database. A study must be registered by the person or organization with overall responsibility. At registration, some data elements are required, while others are optional, and the trial cannot be registered without completion of the required data elements, the approval of an ethics review committee, and compliance of the trial with the requirements of the national health authorities. ClinicalTrials.gov delivers a registry number (NCT), which leading medical journals require for publication of the results. Data updates are required every two years.
We conducted our literature search in January 2015 and extracted data for the period from 2004 to 2011 inclusive, ie, 8 full years. In this way, the authors have 3 years to analyze their data, write and submit an article, and complete the publication process [8-10]. Using the search engine, we selected trials in which the participants were assigned to receive one or more interventions, or no intervention, in order to assess the effects of these interventions on biomedical criteria or state of health. Clinical intervention studies have the highest level of proof. We excluded trials for which there was no information update for more than 2 years, observational studies, registries, and expanded access studies. Intervention studies had to be 'completed', i.e., the study had to have been completed normally, the participants no longer being under observation or treated ('the last subject had been seen', 'the last visit concluded').
The keywords for the literature search were 'dementia or cognitive impairment or alzheimer or parkinson or dementia with lewy bodies or vascular dementia or frontotemporal dementia or mild cognitive impairment', and for the type of intervention 'donepezil or galantamine or rivastigmine or memantine'.
The International Classification of Diseases (ICD) codes for Alzheimer's disease and related disorders are: G30 Alzheimer's disease, G31.0 Frontotemporal dementia, G31.01 Pick's disease (Primary progressive aphasia, Progressive isolated aphasia), G31.09 Other frontotemporal dementia (Frontal dementia) G31.83 Dementia with Lewy bodies (Dementia with Parkinsonism, Lewy body dementia, Lewy body disease), G31.84 Mild cognitive impairment, G31.85 Corticobasal degeneration. ATC (Anatomical Therapeutic Chemical Classification System) codes are: mémantine N06DX01, donépézil N06DA02, galantamine N06DA04, rivastigmine N06DA03.
We started our publication search by examining the 'publication' field of the description of the trial in ClinicalTrials.gov. If an article was flagged, it was checked: name of the investigator, place, correspondence of dates, sponsor, trial conditions, and number of participants enrolled. If no article was found or if an article did not correspond, a manual search was done in Medline with the NCT number. In the absence of an identified article, we completed the search in Medline and Google Scholar with the type of intervention, study conditions, and name of the primary investigator. The articles identified were checked against the information on ClinicalTrials.gov: name of the investigator, place, correspondence of dates, sponsor, trial conditions, and number of participants enrolled. This was done in parallel by two people to limit the number of published articles that were not found.
For all trials we noted the year of registration, the country where the study was conducted, the type of dementia treated, and the funding.
For published trials, we noted dementia stage at inclusion, the number of subjects included, their mean age, the length of the study, its methodology (versus placebo or not), whether it was single- or multicenter, the primary and secondary endpoints, percentage of adverse events and serious adverse events (AE, SAE). The result of the study was classified as positive, doubtful, or negative [11]. It should be noted that in studies of Alzheimer disease and related diseases, in contrast to what is usually recommended, several primary endpoints are often used, in the form of assessment scales. Studies were classified as positive when the primary endpoint or endpoints were positive, whatever the results concerning the secondary endpoints [11]. Studies were classified as negative when none of the primary or secondary endpoints reached significance [11]. Studies were classified as doubtful in the other cases [11]. For unpublished trials, we noted from the initial protocol the aims in terms of number of subjects to be included, length of the study, its methodology (versus placebo or not), whether it was single- or multicenter, and the primary and secondary endpoints.
Of 251 studies from 2004 to 2011 that met the selection criteria, 193 were 'completed', and of these 107 related to our subject (Figure 1). The literature search found published 54 studies [2,12-64], i.e., a publication rate of 50.5%. 83% of the studies were published between 2005 and 2009. Most studies (67%) concerned Alzheimer disease, followed by mild cognitive impairment (8%), Parkinson disease dementia (7.1%) and vascular dementia (5.4%). The pharmaceutical industry helped fund over 85% of the studies, but studies funded from other sources had a higher publication rate (57.6% versus 29.5%).
Flow chart.
The United States conducted the most trials (38% of all trials), followed by Great Britain (9.8%) and Japan (8.9%). In terms of continents, North America accounted for 44.6% of the trials, followed by Europe (33%) and Asia (21.4%). English-speaking countries conducted 61.6% of the trials on the subject. European countries published 75.7% of their studies, English-speaking countries 67%, and Asian countries 37.5%.
The published trials [54] related to donepezil [15], memantine [13], rivastigmine [10], and galantamine [10], and to combinations [6]. The published trials are described in table 1. These trials included a total of 16 769 subjects, with 310 on average per trial (21 trials included between 100 and 500 subjects and 18 trials between 1 and 99 subjects). The mean age of participants ranged from 59 to 89 years, and was between 70 and 79 in 78% of the trials (Figure 2). The mean duration of all published studies was 7.5 months, with a range of 1 to 36 months, most trials lasting 3 to 6 months (Figure 3). In 33 over 54 published studies (61%), the patients present a mild to moderate state of dementia. 11 published studies (20.4%) is including elders with severe dementia. 5 studies are exclusively including people with moderate dementia, and 2 studies are including mild cognitive impairment. The percentages of AE and SAE are very different according to the studies. For AE, the percentage range from 0 to 90%. The reasons of this large variation are: sample size, definition of AE (sometimes, SAE are include in AE; sometimes, AE are only those link to treatment...), reliability of data collection, sometimes lack of data collection. This makes it difficult to compare the percentage of AE between the different studies. IT is the same observation for SAE, the percentage range from 0% to 29%.The primary endpoint sometimes comprised several assessment scales, with cognition the most common criterion (57% of the trials), followed by the clinician's clinical impression (Clinical Interview-Based Impression of Change; CIBIC) (33.3%), adverse drug reactions (16.7%), behavior (14.8%), functional autonomy (13%), and paraclinical variables (11.1%) (Figure 4). By combining the primary and secondary endpoints, the same leading endpoints were found, but not in the same order of importance (Figure 5). For measurement of cognition as primary endpoint, 17 trials used the Alzheimer's Disease Assessment Scale-Cognitive Subscale (ADAS-Cog) [65] and/or the Mini Mental Status Examination (MMSE), 4 trials used the Severe Impairment Battery (SIB), and 9 trials used other scales. Multicenter studies accounted for 87% of trials published.
Number of published trials according to the average age of subjects included.
Number of trials according to the duration of the trial (in month) for published studies and not published studies.
Number of published trials according to primary endpoint (PE) (in abscissa).
Number of published trials according to primary or secondary endpoints (PSE) (in abscissa).
Baseline characteristics of published trials [54].
In terms of results, 25 trials were positive, 25 doubtful, and 4 negative. Depending on the trial methodology, 19 of 28 placebo-controlled trials were classified as doubtful or negative. The others were either single-arm trials designed to assess drug efficacy and safety or to measure paraclinical variables, or parallel-group studies of two formulations or two dosages of a given drug. Of the 9 placebo-controlled trials with a statistically significant result in favor of the drug for all the primary endpoints, 7 assessed cognition (5 using the ADAS-Cog, 1 the MMSE, 1 the SIB). Of the 28 placebo-controlled trials, only 6 lasted one year or more, with 2 negative results and 4 doubtful results.
Of 53 unpublished trials, 17 related to donepezil, 14 to galantamine, 12 to rivastigmine, 8 to memantine, and 2 to a combination. These trials included 12 128 subjects with an average of 242 per trial. The planned duration was unknown in 4 trials. The mean duration of the 49 remaining trials was 8 months, with a range from 2 to 24 months (Figure 3). In terms of outcomes, assessment of cognition was the most common. The figure 6 and figure 7 detail the primary and secondary endpoints. The description of 19 trials did not indicate whether they were single- or multicenter. Of the remaining trials, 79% were multicenter. In terms of methodology, 29 studies were placebo-controlled, and the others were single-arm trials designed to determine drug efficacy and safety or to measure paraclinical variables.
Number of not-published trials according to (in abscissa) primary endpoints (PE).
Number of not-published trials according to primary or secondary endpoints (PSE).
Analysis of pharmacological intervention studies using symptomatic treatments in dementia in the 2000s revealed 107 trials that were completed. These trials were essentially funded by the pharmaceutical industry and related to short-term cognition. Only half of them were published, and those describing a pharmacological intervention versus placebo were most often neither positive nor clinically relevant.
Of 107 interventional studies registered as complete on ClinicalTrials.gov between 2004 and 2011, on donepezil, galantamine, rivastigmine, and memantine in the symptomatic treatment of Alzheimer disease and related diseases, 54 were published, i.e., a publication rate of 50%. This publication rate means that there is a great loss of information. Non-divulgation of a study's findings means that information of scientific value is concealed from the scientific community, patients, and the health authorities [66]. There is a concern that positive results are published while negative results are not [66]. Such bias may skew perception (overestimation) of the treatment effect, notably in meta-analyses, and hence lead the health authorities to take unsuitable or inappropriate decisions regarding care. Failure to publish casts doubt on the scientific integrity of clinical trial sponsors and may discourage patients from participating in trials. How can people be persuaded to take part in a clinical trial when over half of the time no usable information emerges, because it remains unpublished? Also, ethically it is hard to accept that over 12,000 people have been included in trials, and therefore exposed to the risks of clinical research, when there is no end result. And this is an underestimate of the true number of patients exposed to such risks because our study relates only to completed trials, and thus excludes those for which there have been no updates for more than two years and those started and then interrupted. Our results on a specific disease are comparable to those reported by Riveros et al. [67] in a study using ClinicalTrials.gov and to the findings of a recent review of all articles on the subject [68]. Taken together, these results therefore point to a general phenomenon with causes including lack of interest in the results on the part of sponsors or principal investigators [66] and rejection by journals of articles reporting negative results.
The scientific community and the health authorities take the consequences of failure to publish very seriously [69], and the European Medicines Agency (EMA) announced the mandatory publication of the results of all clinical trials from July 2014 onwards.
We wondered about the clinical applicability of trial data, given the age differences between trial participants and patients in the general population, outcomes reduced to psychological and cognitive variables, and, above all, studies limited to six months when, on average, the progression of Alzheimer disease exceeds 10 years. Our results provide no answers.
The mean age of subjects included in research is below 80 in more than three quarters of studies [1]. In most trials, subjects are included from the age of 50, or even 40. The mean age of participants can be between 70 and 80 in studies on the prodromal stage of Alzheimer disease or mild cognitive impairment, but not for the moderate stage onwards.
Dementia and notably Alzheimer disease affect the elderly, the primary risk factor being aging. In France the mean age of patients is 83. While age probably has little influence on the effect of drugs, tolerability can change with age, comorbidities, and co-prescriptions. The risk-benefit ratio possibly differs greatly in elderly populations that are not included in studies but which constitute the great majority of patients receiving the drug. The same is true for acetylcholinesterase inhibitors, the cardiac safety of which is deemed satisfactory in clinical trials but which is probably much less so in "real life".
Outcomes are judged above all by change in cognitive or behavioral scores, and rarely in terms of short-term functional autonomy. These clinical parameters are measured using composite scales the range of which is large. Although the choice of these scales is the object of a professional consensus, their metrological value is often unknown. Furthermore, the clinical relevance of a change in score on these scales has practically never been defined independently, except for the ADAS-Cog. This means that a statistically significant between-group difference in score is hard to translate into the reality of drug efficacy [70-72]. Of the 28 published trials comparing a drug with a placebo, only 9 found a statistically significant result in favor of the drug for all the primary endpoints. Of these 9, 7 assessed cognition (5 with the ADAS-Cog, 1 with the MMSE, 1 with the SIB). The United States Food and Drug Administration (FDA) consider that a difference of at least 4 points on ADAS-Cog is needed to indicate a clinical difference [73]. Yet none of the 5 studies using ADAS-Cog found a difference of 4 or more points between the placebo group and the treatment group at the end of the study. The study using the MMSE found a difference greater than the 1.4 points deemed clinically perceptible by the study sponsors [74]. However, this definition is not considered clinically relevant by other experts questioned independently of any ongoing study [71]. For the study using the SIB, there was a 4.5-point difference between the placebo and treatment groups at the end of the study. In the absence of a consensus, it is difficult to see this difference as clinically pertinent, in as much as the scale ranges from 0 to 100 and its metrological quality has not been studied in depth. These results are consistent with a previous study showing that assessment of the clinical relevance of the immediate action of these drugs is an unresolved issue [75].
As for the long-term effects, they are not always analyzable, because very few trials last a year or longer. Only 6 compared a drug with a placebo, and the results were negative in 2 and doubtful in 4. The primary endpoint was cognition in 3 studies, paraclinical variables in 2, and dependence, adverse drug reactions and clinical global impression in 1. Finally, adverse drug reactions were outcome measures in only 37% of published trials, with most trials lasting 3 to 6 months.
The main limitations of our study are that it is based on data from investigators and on the classification drawn up by the National Library of Medicine in the ClinicalTrials.gov registry. Although it is the largest in the world, this registry does not include all trials conducted worldwide, notably those outside the English-speaking countries. Moreover, we have noted classification errors, which may have affected the queries made with the website's search engine.
From 2004 to 2011, the ClinicalTrials.gov website registered 107 completed interventional studies of symptomatic treatment of Alzheimer disease and related diseases with donepezil, galantamine, rivastigmine, or memantine. Only 50.5% of these 107 trials were published and related primarily to short-term cognition. Their findings do not resolve questions concerning the efficacy of these drugs. What is more, it does not seem that the pharmaceutical industry has attempted to answer the questions of the health authorities concerning the long-term effects and the clinical relevance of these drugs.