That is the message behind numerous publications in recent years, and the COVID-19 pandemic has accelerated that trend.
Dr. Munro Cullum
“The publications have just skyrocketed since 2018, but I think there are still some additional tests that we need to validate using this medium of assessment. Also, I think we need to kind of put on our thinking caps as a field and think outside the box. What novel tests can we develop that will capitalize upon the telehealth environment – interactive tests that are monitoring [the individuals’] performance in real time and giving the examiner feedback, things like that,” said Munro Cullum, PhD, in an interview. Dr. Cullum spoke on the topic at the 2021 Alzheimer’s Association International Conference.
Still, challenges remain, especially factors in the home environment that can adversely affect testing. “Some of our tests are a question-answer, pencil-paper sort of tests that can be well suited to a telemedicine environment, [but] other tests don’t translate as well. So we still have a ways to go to kind of get our test to the next generation when being administered during this type of assessment. But a lot of the verbal tests work extremely well,” said Dr. Cullum, who is a professor of psychiatry at the University of Texas Southwestern Medical Center, Dallas.
Preliminary evidence of equivalence
Some years ago, Dr. Cullum was interested in getting a better understanding of what existing tests could best be performed remotely, and what populations could most benefit from remote assessments. Existing studies were generally supportive of remote testing, but varied significantly in their methodology and design. He went on to publish a study in 2014 showing equivalency of existing tests in the in-person and remote environment, and that helped pave the way for a wave of more recent studies that seem to confirm equivalence of in-person methods.
“If you look at the literature overall, there is a nice, growing body of evidence suggesting support for a host of neuropsychological test instruments. For the most part, almost all have shown good reliability across test conditions,” Dr. Cullum said during the talk.
He said that he is often asked if different test norms will be required for remote tests, but that doesn’t seem to be a concern. “It looks like the regular old neuropsych test norms should serve as well in this remote assessment environment. Although as within hospital testing of patients, conservative use of norms is always an order. They are interpretive guidelines,” he added.
One concern is potential threats to validity within the home environment. He posted an image of a woman at home, taking a remote cognitive test. The desk she sat at overlooked a wooded scene, and had a sewing machine on it. A small dog lay in her lap. “So assessing the home environment, ensuring that it is as close to a clinical standard setting as possible, is certainly advised,” said Dr. Cullum.
Although much progress has been made in studying existing tests in a telemedicine environment, many commonly used tests still haven’t been studied. The risk of intrusions and distractions, and even connectivity issues, can be limiting factors. Some tests may be ineligible for remote use due to copyright issues that might prevent required materials from being displayed online. For those reasons and others, not all individuals are suited for a remote test.
Finally, remote tests should be viewed with healthy skepticism. “In doing clinical evaluations this way, we have to be extra careful to not mis- or overinterpret the findings in case there were any distractions or glitches in the examination that came up during the test,” said Dr. Cullum.