Psychiatr Serv 55:1316-1317, November 2004
© 2004 American Psychiatric Association
Computer-Administered Versus Paper-and-Pencil Mental Health Surveys
To the Editor: Computers are increasingly used to assess mental health and substance abuse treatment outcomes (1). Computerized assessment has the potential to reduce burden on clinical staff, increase the efficiency of data collection and management, reduce the rate of missing data, increase interrater reliability of clinician-administered assessments, eliminate inapplicable items by using branching logic, provide immediate feedback of results, and help identify individuals at risk of poor outcomes in a timely manner that could allow for clinical intervention (2,3). However, it is important to determine whether responses to computerized assessments are comparable to those obtained by paper-and-pencil methods (4).
We conducted a small study to compare feasibility, respondent acceptance, reliability, and mean scores for computerized assessment versus a paper-and-pencil survey. We used a repeated-measures design in which psychiatric outpatients completed two self-report mental health questionnaires by using paper and pencil and by computer. The two survey instruments used were the Polaris Strength Scales, a new 29-item scale focused on goal setting, resiliency, development of emotional and relational skills, and social supports, and the revised Behavior and Symptom Identification Scale (BASIS-24), a 24-item scale that assesses depression and functioning, interpersonal relationships, psychotic symptoms, substance abuse, self-harm, and emotional lability (5). Both instruments are designed for use among persons with serious and persistent mental illness.
The study was conducted over a two-month period in 2002. The sample consisted of 52 English-speaking adults, evenly divided by gender. The age of the participants ranged from 19 to 64 years. A total of 41 participants (84 percent) were Caucasian, 33 (67 percent) were unemployed, and 28 (57 percent) had been in treatment for more than ten years. Two respondents refused to complete the computerized survey. All those who agreed to complete it were able to use the computer after receiving brief (less than five minutes) instructions. More respondents needed help completing the survey by computer than by paper and pencil (eight respondents, or 17 percent, compared with one respondent, or 2 percent;  2=4.28, df=1, p<.05), although respondents preferred the computer by a ratio of 3:1. No significant differences were noted in respondents' evaluation of the time to complete the instruments, understanding of items, clarity of instructions, or any other concerns.
The reliability of ratings made by computer versus by paper and pencil was high, with intraclass correlation coefficients ranging from .83 to .9 for the Polaris Strength Scales and from .82 to .95 for the BASIS-24 domains. Differences in mean scores on the domains assessed were not statistically significant, with one exception: participants reported significantly more frequent psychotic symptoms by paper and pencil than by computer (F=7.19, df=1,44, p=.01).
Limitations of this study include its small sample and possible memory or order effects (that is, whether respondents used the computer first or paper and pencil first), which were not assessed. Self-report of mental health status by computer may enhance quality improvement efforts by improving the capacity to monitor treatment outcomes with greater efficiency than can be achieved with paper-and-pencil methods. Further research is needed to test computerized administration of standardized instruments in larger samples from multiple sites.
Susan V. Eisen, Ph.D.,
Linda L. Toche-Manley, Ph.D. and
Grant R. Grissom, Ph.D.
Footnotes
Dr. Eisen is affiliated with the health services department of the Boston University School of Public Health and the Center for Health Quality, Outcomes, and Economics Research at the E. N. Rogers Memorial Hospital in Bedford, Massachusetts. Dr. Toche-Manley and Dr. Grissom are with Polaris Health Directions in Fairless Hills, Pennsylvania.
References
- Schmitz N, Hartkamp N, Brinschwitz C, et al: Comparison of the standard and the computerized versions of the Symptom Check List (SCL-90-R): a randomized trial. Acta Psychiatrica Scandinavica 102:147–152, 2000[CrossRef][Medline]
- Brown GS, Burlingame GM, Lambert MJ, et al: Pushing the quality envelope: a new outcomes management system. Psychiatric Services 52:925–934, 2001[Abstract/Free Full Text]
- Kobak KA, Schaettle SC, Greist JH, et al: Computer-administered rating scales for social anxiety in a clinical drug trial. Depression and Anxiety 7:97–104, 1998[CrossRef][Medline]
- Guidelines for Computer Based Tests and Interpretation. Washington, DC, American Psychological Association, 1986
- Eisen SV, Normand SLT, Belanger AJ, et al: BASIS-32 and the Revised Behavior and Symptom Identification Scale (BASIS-R), in The Use of Psychological Testing for Treatment Planning and Outcome Assessment, 3rd ed, vol 3. Edited by Maruish M. Mahway, NJ, Lawrence Erlbaum, 2004
Get information about faster international access.
Privacy Policy
Copyright © 2004
American Psychiatric Association.
All rights reserved.
Home
| Search
| Current Issue
| Past Issues
| Subscribe
| All APPI Journals
| Help
| Contact Us
|
