Portions of one of the most commonly used tests to measure cognitive performance in Alzheimer’s disease trials may be too easy and may not accurately assess the range of patients’ cognitive abilities or detect their change over time, according to two complementary studies in the December 11, 2012, Alzheimer’s & Dementia.
Analyses of Alzheimer’s Disease Assessment Scale–Cognitive Behavior Section (ADAS-Cog) scores measured in 193 patients with mild disease who participated in the Alzheimer’s Disease Neuroimaging Initiative (ADNI) in the course of two years detected limitations of the scale that could be improved, reported Jeremy Hobart, MD, PhD, of the Clinical Neurology Research Group, Peninsula College of Medicine and Dentistry, Plymouth, United Kingdom, and his colleagues.
The investigators used observational data from ADNI to show that of 675 measurements made at time points of 0, 6, 12, and 24 months, data from ADAS–Cog total scores spanned the entire range of the scale and had no floor or ceiling effects that would reduce its ability to measure changes and differences in lower-functioning or higher-functioning patients, respectively. However, eight of the scale’s 11 components (all except for word recall, word recognition, and orientation) had statistically significant ceiling effects with a skewed distribution of scores. The mean age of patients was 74 years, 47% were female, and participants had a mean Mini-Mental State Examination score of 23 across all time points.
These results reproduce those that the investigators obtained in a previous psychometric evaluation of the ADAS–Cog in patients who participated in a randomized controlled clinical trial. “These components may underestimate cognitive performance differences in those with mild to moderate Alzheimer’s disease–type dementia,” noted the researchers. “This may lead to problems in detecting clinical change.”
Because often greater than three-fourths of the participants with mild Alzheimer’s disease in the ADNI study scored either 0 or 1 on the majority of ADAS–Cog components, Dr. Hobart and his associates remarked that this would mean that few or no cognitive problems were detected. “However, as there is almost certainly greater variance in patient ability, this finding points to a limitation in the ADAS–Cog score function—namely that the ADAS–Cog, in its current form, is not subtle enough to record and monitor variance in the mildest stages of Alzheimer’s disease–type dementia.”
In a second study that analyzed the extent to which the ADAS–Cog accurately measured cognitive performance in the same sample of patients, the investigators tried to avoid the limitations imposed by classical means of assessing the reliability and validity of scales. They used a method called Rasch measurement theory, which is a mathematical model designed to analyze the extent to which the rating scale data meet certain conditions necessary for the scale to record accurate measurements of cognitive performance. It provides diagnostic information that can help to revise a scale by exposing anomalies in the scale that can be improved and then retested, Dr. Hobart and his colleagues explained.
The range of cognitive performance measured by the 11 ADAS–Cog components suboptimally targeted the range of cognitive performance observed in patients in the sample. In six components, the integer-based scoring method used to assign cognitive performance did not reflect a continuum of performance as it was intended to, but instead indicated that some component scores were much more likely to occur than others, meaning that a higher score on one of these components did not confirm more cognitive impairment.
These gaps in some of the components’ abilities to measure cognitive performance means that their precision is limited, and that the raw scores of the ADAS–Cog and the linear relationship they are thought to reflect is actually an S-shaped relationship in which a one-point change in ADAS–Cog measurement varies across the range of the scale, such that it is highest at the extremes of its range and lowest at the center of its range, the investigators said.
—Jeff Evans
IMNG Medical News