Psychiatr Serv 53:1322-1324, October 2002
© 2002 American Psychiatric Association
Utility and Limits of the Mini Mental State Examination in Evaluating Consent Capacity in Alzheimer's Disease
Scott Y. H. Kim, M.D., Ph.D. and
Eric D. Caine, M.D.
|
Abstract
|
The call for formal capacity assessments of potential research participants with impairments due to illnesses such as Alzheimer's disease is increasing. Because such an evaluation of every potential subject requires significant resources, clinicians and researchers may want to know the utility and the limits of a familiar cognitive screening test, the Mini Mental State Examination (MMSE), in capacity evaluations. In this study of 37 patients with mild to moderate Alzheimer's disease, a fairly wide range of MMSE scores (21 to 25, which includes an often used cutoff for "normal") did not discriminate capacity status well. Nevertheless, there may still be approaches that enhance the practical utility of the MMSE in capacity assessment.
|
Introduction
|
The competence of persons with Alzheimer's disease and other cognitive disorders to give consent to participate in research studies continues to raise ethical concerns, including calls for formal competency assessments of potential research participants (1). But performing a full, formal capacity evaluation for every potential subject can be time-consuming and expensive. Clinicians and researchers may want to know whether a familiar, widely used test—the results of which are usually already available—could inform them of the likely results of a full capacity examination.
The Mini Mental State Examination (MMSE) (2) is widely taught and used among health care professionals, serving as a universal, if limited (3), indicator of severity of impairment among persons with Alzheimer's disease and other cognitive disorders. Several researchers have explored the relationship between MMSE scores and consent capacity, but the results have been mixed (4). The studies used diagnostically heterogeneous populations, focused on treatment consent capacity, and used different criterion standards for capacity.
In this study we prospectively examined the limits and the utility of the MMSE as a screening tool for assessing the capacity of patients with well-defined Alzheimer's disease to consent to participate in research studies.
|
Methods
|
Participants
Our study participants were recruited from the outpatient geriatric neurology and psychiatry clinic at the Monroe Community Hospital, an affiliate of the University of Rochester, between May 1999 and April 2000. Potential participants were persons who had a diagnosis of probable Alzheimer's disease by the criteria of the National Institute of Neurological and Communicative Disorders and Stroke and the Alzheimer's Disease and Related Disorders Associations (NINCDS-ADRDA criteria) and had received an MMSE score of 18 or above in the preceding 12 months, although at the time of the study some patients had an MMSE score below 18. The cutoff was chosen to identify participants who were capable and incapable of providing consent (5). MMSE scores range from 0 to 30, with lower scores indicating greater impairment.
Measures
The MacArthur Competence Assessment Tool, Clinical Research version (MacCAT-CR) (6) is a semistructured interview that measures performance on legally relevant standards for competence with 21 items, each scored on a 3-point scale from 0 to 2, indicating, respectively, inadequate, partial, or adequate. The items reflect Appelbaum and colleagues' (7) four-abilities model of competence: the ability to understand factual information (13 items), the ability to appreciate and apply the facts to one's own situation (three items), the ability to reason and manipulate the facts appropriately (four items), and the ability to communicate a choice (one item). In this report, only the data on the first three more substantive standards are presented.
The interview portrayed a hypothetical, randomized, double-blind, placebo-controlled clinical trial for a new medication for Alzheimer's disease. The MacCAT-CR has been shown to have excellent content validity and good inter-interviewer and interscorer reliabilities (7,8,9). The version used in this study can be obtained from the first author.
We categorized participants as either capable or incapable on each of the three standards on the basis of validated cutoff scores of 18 of 26 for understanding, 5 of 6 for appreciation, and 6 of 8 for reasoning (8). Because there is no consensus yet on which legal standard to use in the research consent setting, a conservative categorization method of combining all three standards was added to the three individual standards by which participants were categorized as incapable if they failed any of the three individual standards. Thus each subject was categorized as capable or incapable according to four different criterion standards. The MMSE was administered on the same day as the capacity interview.
The study was approved by the research subjects review board of the University of Rochester, and written informed consent was obtained from the participants or their surrogates.
Statistical analysis
We performed a receiver operating characteristic (ROC) analysis with SPSS 10.0. This analysis plots the sensitivity of a test against the value of 1 minus specificity, where sensitivity is the ability of a test to identify a true case (a true positive) among all cases and specificity is the ability to identify a true noncase (a true negative) among all noncases. In this study, a true positive was defined as an incapable participant, as determined by one of the four criterion standards, who scored at or below a given MMSE score. The 95 percent confidence intervals for areas under the ROC curves were calculated.
|
Results
|
The study group comprised 37 persons. Sixty percent were women. The mean±SD age was 78.7±5.8 years, the mean number of years of education was 13.8±3.2, and the mean MMSE score was 22.9±3.8 (range, 16 to 28).
Table 1 lists the areas under the ROC curves and their 95 percent confidence intervals for the MMSE as a test for identifying incapacity against the three criterion standards established by the MacCAT-CR or the combined standard. None of the confidence intervals include .5—the expected area if the null hypothesis were true—indicating that the MMSE significantly adds to the identification of incapacity. However, this significant effect is modest: the areas are relatively small, and no cutoff point yields both high sensitivity and high specificity.
View this table:
[in this window]
[in a new window]
|
Table 1. Sensitivity and specificity of the Mini Mental State Examination in identifying, according to four standards, the capacity status of 37 persons with mild to moderate Alzheimer's diseasea
a Only observed MMSE scores are included. Each subject was deemed capable or incapable on each of the three legal standards as well as by combining all three standards. The three legal standards are based on the patients' scores on the MacArthur Competence Assessment Tool, Clinical Research version. The combined standard defines as incapable any subject deemed incapable by any of the three legal standards. Sensitivity is defined as the number of incapable subjects scoring the indicated MMSE cutoff score or below divided by the number of incapable subjects for a given standard, in percentages. Specificity is defined as the number of capable subjects scoring above the indicated MMSE cutoff score divided by the number of capable subjects for a given standard, in percentages.
|
High specificity, in the range of 85 to 94 percent, was approached at MMSE scores of 19 or less. Such a cutoff score correctly identified 39 to 54 percent of the incapable subjects. High sensitivity, ranging from 91 to 100 percent, was approached at an MMSE score of 26 while a specificity of 29 to 36 percent was retained.
|
Discussion and conclusions
|
Past attempts to define the utility of the MMSE in capacity assessments have focused on treatment consent in heterogeneous patient groups, with mixed results (4). Our study is the first to assess the screening utility of the MMSE in a group with well-defined Alzheimer's disease, using a validated capacity instrument specifically designed for research on informed consent, the MacCAT-CR, as a benchmark.
Within a certain range of scores, the MMSE was at best a modest discriminator of decisional capacity in this study group. This range, 21 to 25, includes the cutoffs often used for "normal" for the MMSE. Given that abilities relevant to competence are correlated with executive functions (10), the modest discriminatory power of the MMSE may reflect the instrument's relative insensitivity in detecting executive dysfunction.
Do these findings imply that the MMSE has no role in screening for incapacity? The utility of a test depends partly on its purpose. For instance, for a nontherapeutic, greater-than-minimal-risk study, a researcher may have strong ethical reasons to select only patients who are clearly capable of providing consent. Such a researcher would be advised to use a higher MMSE cutoff score—for example, a cutoff of 26 has a sensitivity of 91 to 100 percent in identifying persons who are incapable of providing consent—as a prelude to a more comprehensive task-specific evaluation. Alternatively, if the goal were to identify as many capable subjects as possible, even at considerable expense of effort and time, then a cutoff of 19, which has a specificity range of 85 to 94 percent, might be used. Anyone who scores below that level is very likely to be deemed incapable of providing consent.
Thus, although the MMSE is not a very good predictor of incapacity, our findings suggest how best to extract the modest amount of utility the instrument offers—with virtually no additional effort, because the scores are already available for most patients with Alzheimer's disease.
Our study is limited by its small sample, the use of a hypothetical research scenario to assess capacity, and its time-slice view of complex phenomena that may fluctuate over time. Competency judgments also take into account important contextual issues, such as the risk-benefit calculus (1). In this study, we used a scenario for a placebo-controlled, double-blind, randomized trial for a new treatment for Alzheimer's disease. Thresholds for competence cannot be assumed to be equal across research studies with different risk-benefit ratios. Finally, because sensitivity and specificity are features of tests and not of populations, the true usefulness of the MMSE as a screening device will depend on the prior probability of incapacity in a particular situation.
|
Acknowledgments
|
This research was supported in part by grants MH-18911 and MH-64172 from the National Institute of Mental Health and by the American Association for Geriatric Psychiatry Eisai-Pfizer Alzheimer's Disease Research Fellowship awarded to Dr. Kim. The authors thank Colleen McCallum, M.S.W., Pierre Tariot, M.D., and Yeates Conwell, M.D.
|
Footnotes
|
The authors are affiliated with the department of psychiatry of the University of Rochester School of Medicine and Dentistry, Box PSYCH, 300 Crittenden Boulevard, Rochester, New York 14642 (e-mail, scott_kim{at}urmc.rochester.edu). Dr. Kim is also director of the program in clinical ethics at the University of Rochester Medical Center.
|
References
|
- Research Involving Persons With Mental Disorders That May Affect Decisionmaking Capacity, vol 1. Rockville, Md, National Bioethics Advisory Commission, 1998
- Folstein MF, Folstein SE, McHugh P: Mini-Mental State: a practical guide for grading the cognitive state of patients for the clinician. Journal of Psychiatric Research 12:189-98, 1975[CrossRef][Medline]
- Clark CM, Sheppard L, Fillenbaum GG, et al: Variability in annual Mini-Mental State Examination score in patients with probable Alzheimer disease: a clinical perspective of data from the Consortium to Establish a Registry for Alzheimer's Disease. Archives of Neurology 56:857-862, 1999[Abstract/Free Full Text]
- Kim SYH, Karlawish JHT, Caine ED: Current state of research on decision-making competence of cognitively impaired elderly persons. American Journal of Geriatric Psychiatry 10:151-165, 2002[Abstract/Free Full Text]
- Marson DC, Ingram KK, Cody HA, et al: Assessing the competency of patients with Alzheimer's disease under different legal standards: a prototype instrument. Archives of Neurology 52:949-954, 1995[Abstract]
- Appelbaum P, Grisso T: MacCAT-CR: MacArthur Competence Assessment Tool for Clinical Research. Sarasota, Fla, Professional Resource Press, 2001
- Appelbaum PS, Grisso T, Frank E, et al: Competence of depressed patients for consent to research. American Journal of Psychiatry 156:1380-1384, 1999[Abstract/Free Full Text]
- Kim SYH, Caine ED, Currier GW, et al: Assessing the competence of persons with Alzheimer's disease in providing informed consent for participation in research. American Journal of Psychiatry 158:712-717, 2001[Abstract/Free Full Text]
- Carpenter WT Jr, Gold J, Lahti A, et al: Decisional capacity for informed consent in schizophrenia research. Archives of General Psychiatry 57:533-538, 2000[Abstract/Free Full Text]
- Marson D, Harrell L: Executive dysfunction and loss of capacity to consent to medical treatment in patients with Alzheimer's disease. Seminars in Clinical Neuropsychiatry 4:41-49, 1999[Medline]
This article has been cited by other articles:
|
|
|
|
 
S. Ripley, S. Jones, and A. Macdonald
Capacity assessments on medical in-patients referred to social workers for care home placement
Psychiatr. Bull.,
February 1, 2008;
32(2):
56 - 59.
[Abstract]
[Full Text]
[PDF]
|
|
|
|
|
|
|
P. S. Appelbaum
Assessment of Patients' Competence to Consent to Treatment
N. Engl. J. Med.,
November 1, 2007;
357(18):
1834 - 1840.
[Full Text]
[PDF]
|
|
|
|
|
|
|
R. Gregory, F. Roked, L. Jones, and A. Patel
Is the degree of cognitive impairment in patients with Alzheimer's disease related to their capacity to appoint an enduring power of attorney?
Age Ageing,
September 1, 2007;
36(5):
527 - 531.
[Abstract]
[Full Text]
[PDF]
|
|
|
|
|
|
|
L. B. Dunn, M. A. Nowrangi, B. W. Palmer, D. V. Jeste, and E. R. Saks
Assessing Decisional Capacity for Clinical Research or Treatment: A Review of Instruments
Am J Psychiatry,
August 1, 2006;
163(8):
1323 - 1334.
[Abstract]
[Full Text]
[PDF]
|
|
|
|
|
|
|
B. W. Palmer, L. B. Dunn, P. S. Appelbaum, S. Mudaliar, L. Thal, R. Henry, S. Golshan, and D. V. Jeste
Assessment of Capacity to Consent to Research Among Older Persons With Schizophrenia, Alzheimer Disease, or Diabetes Mellitus: Comparison of a 3-Item Questionnaire With a Comprehensive Standardized Capacity Instrument
Arch Gen Psychiatry,
July 1, 2005;
62(7):
726 - 733.
[Abstract]
[Full Text]
[PDF]
|
|
|
|
|
|
|
J.H.T. Karlawish, D. J. Casarett, B. D. James, S. X. Xie, and S. Y.H. Kim
The ability of persons with Alzheimer disease (AD) to make a decision about taking an AD treatment
Neurology,
May 10, 2005;
64(9):
1514 - 1519.
[Abstract]
[Full Text]
[PDF]
|
|
|
|
|
|
|
R. B Scott, E. Farmer, A. Smiton, C. Tovey, M. Clarke, and K. Carpenter
Methodology of neuropsychological research in multicentre randomized clinical trials: a model derived from The International Subarachnoid Aneurysm Trial
Clinical Trials,
February 1, 2004;
1(1):
31 - 39.
[Abstract]
[PDF]
|
|
|
Get information about faster international access.
Privacy Policy
Copyright © 2002
American Psychiatric Association.
All rights reserved.
Home
| Search
| Current Issue
| Past Issues
| Subscribe
| All APPI Journals
| Help
| Contact Us
|
TOP
Abstract
Introduction
