Professional psychologists are challenged to determine the appropriate use of interactive computer therapy programs. Although such programs have the potential of enhancing delivery of mental health services and reaching ever broader audiences, they raise serious clinical, legal, ethical, and practical concerns. This article reports on a controlled clinical trial comparing short-term traditional individual therapy with a computer-based intervention overseen by a therapist. Results were favorable and comparable in both conditions, with individual therapy outperforming computer-based therapy on some measures. The practitioner's use of computer-based psychotherapy interventions is discussed and some guidelines offered.

Computer technology is changing the face of psychotherapy. Should interactive programs be used as an adjunct to treatment? Should they be recommended for use without other treatment? If so, for what diagnostic groups and with what precautions should they be recommended? Should some clients explicitly be warned against unsupervised interventions? An American Psychological Association task force (Nickelson, 1997) report has cited a minefield of legal, ethical, and financial issues of concern to psychologists interested in these new technologies.

MARION K. JACOBS received her PhD in clinical psychology from the University of Southern California in 1969. She was coordinator of the University of California, Los Angeles (UCLA), psychology clinic for 19 years and is now adjunct professor emerita and is in independent practice in Laguna Beach, CA. Her research interests are in modes of delivering psychological assistance.
ANDREW CHRISTENSEN received his PhD in clinical psychology from the University of Oregon in 1976. He is professor of psychology at UCLA. His research interests are in outcome research, couple therapy, couple conflict, and nonprofessional intervention.
JOHN R. SNIBBE received his PhD in clinical psychology from the University of Utah in 1970. He is now director of the Children's Crisis Team at Harbor/UCLA Medical Center and assistant clinical professor of child psychiatry at UCLA.
SHARON DOLEZAL-WOOD received her PhD in clinical psychology. She is now in independent practice in Los Angeles, CA.
ALICE HUBER received her PhD from UCLA in 1994. She is clinical research psychologist at the Matrix Institute and the Los Angeles Addiction Research Consortium. Her research interests are behavioral and biological treatments for substance dependence in adults and adolescents.
ALEXANDRA POLTEROK received her bachelor's degree from UCLA and was a research assistant on this project.
CORRESPONDENCE CONCERNING THIS ARTICLE should be addressed to Marion K. Jacobs, 2060 Temple Hills Drive, Laguna Beach, California 926512660. Electronic mail may be sent to mkjacobsI@home.com.

Internet users can access real-time counseling services, receive e-mail counseling advice, or log on to professionally assisted chat rooms, self-help groups, and bulletin boards for everything from borderline personality problems to panic attacks. As with the self-help book movement, the quality of the offerings varies widely, as does professional reaction to the prospect of widespread use of computer and computer-assisted therapy (Newman, Consoli, & Taylor, 1997). Outcome and consumer satisfaction remain largely unevaluated.

Computer-Based Versus Traditional
Psychotherapy Project

The present study examines feasibility, efficacy, and client satisfaction issues with regard to using an interactive computer program for delivery of mental health services. The goal is to provide practitioners and program managers with data, as well as to raise issues, that can assist them in deciding if, when, and how to incorporate electronic programs into their work. Short-term, therapist-delivered treatment is compared with treatment provided primarily by means of an interactive software program with oversight by a therapist.

The Therapeutic Learning Program (TLP; Gould, 1989) was chosen as the experimental intervention for this study because it is a generic, interactive computer program that allows the client to deal with any psychological problem as long as the goal can be behaviorally specified. It was designed as an adjunct to traditional

1

psychotherapy, although in the present study, it was used as the primary intervention. TLP is grounded in Gould's theory of adult development (1978) and represents an integration of psychodynamic and cognitive-behavioral concepts and strategies. The 10-session TLP program was presented to the user as a psychoeducational experience, creating a learning laboratory that afforded the user an opportunity to explore a problem area and develop possible coping strategies with the option of implementing behavior change. A series of comprehensive menus in each of the 10 sessions guided the client in identifying a changeable problem, formulating a desired action plan, and working on issues that might block implementation of the plan. At the completion of each session, a summary printout of client responses was generated.

Prior studies using TLP as an adjunct to professionally run therapy groups with psychiatric patients in large HMOs (DolezalWood, Belar, & Snibbe, 1998; Talley, 1987) showed both patient satisfaction and improvement. The current study went further by testing TLP as an individual intervention with sharply limited therapist involvement in the delivery of the treatment.

Ninety clients presenting with a variety of problems and symptoms were randomly assigned to an individual therapy condition, in which they received 10 weeks of individual, focused psychotherapy, or to a TLP condition, in which they received 10 weekly TLP sessions with brief therapist contact. Outcome was evaluated on a variety of therapist- and client-completed measures of symptom occurrence, general functioning, and satisfaction.

Participants were recruited from the community through newspaper advertisements, had to be at least 18 years of age, and could have any presenting problems except drug and alcohol abuse, severe mental disorders uncontrolled by medication, or dementia. The study was conducted at the University of California, Los Angeles (UCLA) psychology training clinic. Cost for participating in the program was $100 for 10 sessions. Up to $50 was refundable, contingent upon the completion of the 10 sessions, the posttreatment assessment measures, and the 6-month follow-up measures. Twenty-five dollars each was refunded after the posttreatment and follow-up assessments.

The 90 adults (37 male, 53 female) who agreed to participate in the study ranged in age from 19 to 84, with the average age of 44. Thirty-six were single, 24 married, 4 separated, 20 divorced, and 6 widowed. The primary diagnoses were as follows: affective disorder, 24; anxiety disorder, 4 adjustment disorder, 18; V-code, 41; other, 3. Average years of education was 16.6. Overall, this sample was probably less pathological than a general outpatient population.

Following phone screening, clients were scheduled for an in-person 2-hr intake appointment, which was not included as part of the 10 sessions. During the first part of the intake, clients completed a sociodemographic questionnaire, a medical screening form, an informed consent form, and the Mini-Structured Clinical Interview for DSM IIIR (Mini-SCID; First, Gibbon, Williams & Spitzer, 1996), a diagnostic assessment tool that was administered by computer. The instrument provides information on the possible presence of serious psychopathology. On the basis of information provided by the Mini-SCID, therapists then interviewed clients to gather further background and diagnostic information. made a diagnosis, and determined the client's eligibility to participate. At this session, clients completed the Minnesota Multiphasic Personality Inventory-I (MMPI-1) and a battery of standardized question-

naires, discussed below, that assessed depression, anxiety, perceived stress, and other psychiatric symptoms; therapists also assisted clients in identifying three presenting problems that served as their target complaints.

The clients in the individual therapy condition met weekly with a therapist for ten 50-min sessions of problem-focused, eclectic psychotherapy. We wanted this condition to represent routine or standard treatment in a short-term clinic but in its best form. In the computer-based condition, TLP guided clients through a 10-session, weekly, personal-problem-solving sequence. The therapist's role during TLP was to (a) instruct on the mechanics of using the TLP package, (b) review the summary printout after each session to ensure that the client remained focused on a clearly specified problem and followed TLP directions, and (c) check post-session printouts for any indications of current clinical crisis. Although client questions were addressed, therapists kept contact limited and focused on the computer therapy program. Average time of contact, including all instruction and housekeeping details, was 20 min per session, with more contact during earlier sessions. The computer portion took, on average, 32 min to complete.

Master's level, post-internship graduate students in the PhD program in clinical psychology at UCLA served as therapists in both conditions. Two licensed, clinical psychologists, who were not involved in the research project, were selected as supervisors.

At the end of the 10 sessions, clients again completed a full battery of outcome measures similar to those collected at intake. In addition, they completed a measure of their satisfaction with the therapy. Six months after completion of the 10 sessions, all clients were contacted by mail or phone and were sent these measure to complete again.

Four kinds of measures were used to assess outcome: standardized client measures, individualized client measures, therapist-completed measures, and client satisfaction measures. Standardized client measures of outcome included (a) the Beck Depression Inventory (Beck, Rush, Shaw, & Emery, 1979), a 21-item self-report measure of depression symptoms; (b) the Perceived Stress Scale (Cohen, Kamarch, & Mermelstein, 1983), a 14-item measure of perceived stressful life situations; (c) the State-Trait Anxiety Inventory (Spielberger, Gorsuch, & Luchene, 1970), a measure composed of two separate self-administered, self-report scales of 20 questions each, measuring state anxiety(i.e., how one feels now) and trait anxiety (i.e., how one generally feels); and (d) the Brief Symptom Inventory (Derogatis & Melisaratos, 1983), a 53-item self-report inventory that yields nine symptom scores and three global summary scores. For the current study, one of the summary scores, the Global Severity Index was used as a general index of psychological distress. For each of these measures, higher scores indicate greater problems.

In addition to these standardized measures, clients specified three current problems of most concern to them on a target complaints measure and made ratings about each of those problems. One question asked the client to rate how much the problem bothered him or her on a 13-point scale that ranged from 1 (not at all) to 13 (couldn't be worse). A second question, included at post-treatment assessment and follow-up but not at the pretreatment assessment, asked the client to rate how much the problem had changed "since you began treatment." This rating was made on a 9-point scale that ranged from 1 (the problem has gotten worse) to 9 (the problem has gotten much better). An Average Bother

2

score was generated by averaging ratings for the bother scale across the client's identified complaints. The Average Change score was computed by averaging ratings for change across the client's complaints.

The last of the client measures was an 8-item Client Satisfaction Questionnaire, which is part of the larger 18-item Service Evaluation Questionnaire (Attkisson & Zwick, 1982). On 4-point scales that range from excellent to poor, clients rate the effectiveness and their satisfaction with the services they received.

In addition to client measures, therapists completed two measures of client functioning. The Global Assessment of Functioning from the Diagnostic and Statistical Manual of Mental Disorders (3rd ed., rev.; American Psychiatric Association, 1987) is a single item rating made by a therapist in an attempt to quantify a patient's level of symptomatology and functioning. The range is 1-90, from most symptomatic to least symptomatic. The Therapist Rating of Patient Functioning is a 9-item questionnaire created by the research team, on which the therapist rates on a 7-point scale from very, poor to excellent, the client's affective state, motivation for change, ability to express feelings, self-awareness, ability to use therapy and implement decisions, and interpersonal relations and communication. ¹

Of the 90 participants who began the study, 4 from the TLP condition dropped out of treatment, all within the first 4 sessions. All of the remaining 86 participants completed the treatment program and the posttreatment assessments. One of the dropouts completed some of the post-treatment assessment measures, and the data are included below. Six months after treatment termination, over 90% of those who completed treatment provided follow-up data: 41 of the 45 individual therapy participants and 37 of the 41 TLP participants. Eight could not be assessed for various reasons (e.g., relocation, death).

The means and standard deviations of all the outcome measures are presented in Table 1 for the two treatment conditions (TLP vs. individual therapy) at each of three assessment points: pretreatment, posttreatment, and 6-month follow-up. Because of the possibility that treatment effects might interact with clients' level of distress, an MMPII indicator of pathological distress (the Welsh-A ratio; Welsh, 1956) was used as a factor in all analyses.

We conducted analyses of variance (ANOVAs) on all measures in order to examine the effects of treatment condition (TLP vs. individual therapy), the effects of time (pretreatment vs. post-treatment vs. follow-up), the effects of distress (less distress vs. more distress), and any interactions between these factors². Client scores on the Beck Depression Inventory Perceived Stress Scale, State-Trait Inventory, and Brief Symptom Inventory all indicated statistically significant reductions in the severity of symptoms from pre-to posttreatment and a significant effect of distress, with more distressed clients scoring worse than less distressed clients, but with no main effects of treatment condition or interactions between treatment and time (the two therapy conditions were not differentially effective). When we looked at changes over time for those clients who completed the 6-month follow-up as well as pretreatment and posttreatment assessments, we found that scores at the 6-month follow-up assessment were significantly better than the pretreatment scores for all the client standardized measures except the Brief Symptom Inventory. However, the follow-up scores were not significantly better than the posttreatment scores. Furthermore, the State-Trait Anxiety Inventory showed a significant increase in

¹The research team consisted of Marion K. Jacobs, Andrew Christensen, John R. Snibbe, Sharon Dolezal-Wood, and Phillip Akutsu.

3

anxiety from posttreatment to follow-up. As in the earlier analyses, there was an effect for distress level but no significant effect of treatment condition or interactions between treatment condition and time (no differential effect of the treatments).

On the two therapist measures -- the Global Assessment of Functioning Scale and the Therapist Rating of Patient Functioning -- a repeated measures ANOVA indicated that posttreatment scores were significantly higher than pretreatment scores. On the Therapist Rating of Patient Functioning, individual therapy had significantly higher ratings than TLP. However, this finding was due in part to a pretreatment difference in favor of individual therapy versus TLP. There were no other significant main effects or interactions.

There were two individualized measures of client change, Average Bother ratings and Average Change ratings. Analyses of the Average Bother ratings revealed a significant effect of time both when examining two levels of time (pre- to posttreatment) and three levels of time (pretreatment, posttreatment, and follow-up). Specific contrasts on the later analysis indicated that scores at 6-month follow-up were significantly lower than at pretreatment but that there was no significant change from posttreatment to 6-month follow-up. On this analysis, there was a significant effect of distress, but there were no other significant main effects or interactions.

Since the Average Change ratings could not be completed at pretreatment assessment, an ANOVA was conducted on posttreatment assessment scores. Factors were treatment condition (TLP vs. individual therapy) and level of pathology (more distressed vs. less distressed). There was a significant effect of treatment, with participants in the individual therapy condition evidencing more change than those in the TLP condition. There were no other significant effects.

We also conducted an ANOVA with the addition of a time factor (posttreatment vs. follow-up). There was a main effect of distress level and treatment condition. More distressed participants had less change than less distressed participants; participants in the individual therapy condition evidenced more change than those in the TLP condition. There were no other significant effects.

We wanted to get some measure of the clinical significance of our findings. Because of the diversity of the clients and their problems, no single standardized measure, such as the Beck Depression Inventory, was an adequate indicator of presenting problems. Although some clients had presenting problems of depression. others did not. Therefore, the target complaints measure, because it assessed the major specific problems of each client, was the best candidate for a measure of clinical significance. We selected a 6 or above on the 9-point Average Change scale as an indicator of clinically significant change. A score of 6 indicates that the problem is more than a little better; a score of 9 indicates that the problem is much better.

We looked first at the target complaints that changed the most. The great majority of clients achieved our level of clinically significant change on at least one of the complaints (76% in the TLP condition and 91% in the individual therapy condition). However, when looking at the average level of change across all target complaints, 52% of the TLP condition and 73% of the individual therapy condition reached an average level of clinically significant change across all of the target complaints. A chi-square test examining this difference was statistically significant, suggest-

ing greater change in the individual therapy condition, X²(1, N = 87) = 4.1, p < .05.

On our measure of client satisfaction -- the Client Evaluation of Services -- ANOVAs similar to those above revealed that individual therapy clients were more satisfied than TLP clients and that clients were more satisfied at posttreatment than at follow-up.

Thus, our study shows that computer-based therapy generated improvement from pretreatment to posttreatment on a variety of important psychological measures. In addition, the evidence suggests that most of these changes persisted through a 6-month follow-up period, albeit with some regression. It is common for some of the effects of treatment to decline over time. Further-more, on most of our measures, the changes in computer-based treatment were not distinguishable from the changes generated in traditional individual psychotherapy. Said another way, even though the computer treatment involved less than half as much therapist time as individual psychotherapy, it generated similar effects.

There are two important cautions. First, our data suggest that individual therapy outperformed the computer-based therapy on several measures. Clients were more satisfied with individual therapy and performed better on measures of targeted change. Thus, computer-based therapy, though positively valued by clients, is not as efficacious, nor as well liked, as individual therapy. Given this, along with our general professional concern about users who may be in crisis, we would be hesitant to recommend computer-based treatments in the absence of any therapist oversight.

Second, ours was generally not a sample of seriously disturbed people. Even though all had problems, for which they were willing to seek paid help, many were not diagnosable. TLP worked as well with the more distressed participants in our study as with the less distressed participants, but given the limitations of our sample, research is needed to determine the extent of TLP's applicability to patients with severe pathology.

Just as electronic therapy packages have potential value, they also have potential pitfalls. On the positive side, these therapeutic packages take advantage of the ability of computers to store and present information interactively in a variety of ways, based on the user's particular needs. Further, such packages are reusable, can be made widely available at a relatively low cost, offer clients the convenience of a therapy session by simply slipping a disk into the computer or logging on to the Internet, and have the potential of reaching populations that otherwise are unlikely to obtain psychological help. However, these packages raise many serious, and as yet unanswered, legal and ethical questions regarding the effects of such interventions and the responsibilities of the people who create or promote them (Barak, 1999).

Unlike self-help books or self-help audio and video tapes, computer therapy programs interact with the user in a manner somewhat analogous to a therapist -- that is, the program requests personal information and tailors its responses to the answers given by that person. Because this kind of format encourages experiential engagement, involves direct interaction, and probably generates greater expectation for help by the user, we believe computer therapy programs require a higher level of professional control than other self-help materials.

4

We find it useful to think about computer therapy programs along a continuum of professional therapy assistance. At one extreme, there is computer treatment alone without any live therapist contact. In fact, the author of TLP has recently developed a version that is available on the Internet for a relatively small fee (currently $30; www.masteringstress.com). Next, there is therapist-assisted computer treatment. In this case, the computer treatment is the primary treatment, but the therapist is available to assist with the computer treatment and to be available for clinical emergencies. Moving toward greater professional involvement, there is computer-assisted therapist treatment. Here, the individual therapist is central to the treatment, but the computer program assists the therapist's work. Originally, TLP-with its summary printouts of each session for therapist and client -- was designed to be used in this way. Finally, there is individual therapy without any computer assistance (i.e., traditional individual therapy).

Clinicians or their assistants could monitor and supervise clients on programs such as TLP, allowing them to serve more clients than if they only conducted traditional one-to-one psychotherapy. Less disturbed clients might use programs such as TLP with minimal therapist assistance, allowing therapists to devote more time to their more disturbed clients.

Endorsing the use of interactive computer programs on strictly self-help basis is more problematic. As noted by Barak (1999), in an excellent review of psychological applications on the Internet, a major reservation about strictly self-help use of computer mental health interventions is the lack of editorial control and lack of a review process for the materials being offered. The difficulty of examining the licensing and certification of Internet-based providers of psychological information provides an opportunity for charlatanism and harm to the public on the one hand and damage to the integrity of professional psychology on the other. The felt need for some oversight is reflected in the recent formation of the International Society for Mental Health Online (see http://www.ismho.org for more information) to help protect both the public and professionals from potentially damaging activity.

Computer therapy offerings are burgeoning, just as self-help books and self-help groups have in the past. Ideally, such packages would all be well crafted and researched by qualified professionals. The reality is, however, that most are not. Even with the best of packages, though, our data suggest that for enhanced outcome, therapist involvement is important. Certainly in terms of crisis response, therapist involvement is essential. We urge psychologists to be receptive to interactive computer approaches, viewing them as a new tool in our armamentarium to be used creatively in conjunction with professional assistance.

References

American Psychiatric Association. (1987). Diagnostic and statistical manual of mental disorders (3rd ed., rev.). Washington, DC: Author.

Attkisson, C. C., & Zwick, R. (1982). The Client Satisfaction Questionnaire: Psychometric properties and correlations with service utilization and psychotherapy outcome. Evaluation and Program Planning, 5, 233-237.

Barak, A. (1999). Psychological applications on the Internet: A discipline on the threshold of a new millennium. Applied & Preventive Psvchology, 8,231-245.

Beck, A. T., Rush, A. J., Shaw, B. F., & Emery, G. (1979). Cognitive therapy qf depression. New York: Guilford Press.

Cohen, S., Kamarch, T., & Mermelstein, R., (1983). A global measure of perceived stress. Journal of Health and Social Behavior, 24, 385-396.

Derogatis, L. R., & Melisaratos, N. (1983). The Brief Symptom Inventory: An introductory report. Psychological Medicine, 13, 595-605.

Dolezal-Wood, S. L., Belar, C. B., & Snibbe, J. (1998). ). A comparison of computer-based psychotherapy and cognitive-behavioral therapy. Journal of Clinical Psychology in Medical Settings, 5, 10.3-115.

First, M., Gibbon, M., Williams, J., & Spitzer, R. (1996). Mini-Structured Clinical Interview for DSM III-R Screen Patient Questionnaire [Computer software). North Tonawanda, NY: Multi-Health Systems.

Gould, R. L. (1978). Transformarions: Growth and change in adult life. New York: Simon & Schuster.

Gould, R. L. (1989). Therapeutic Learning Program (Version 5.0) [Computer software]. Santa Monica, CA: Interactive Health Systems.

Marks, I., Shaw, S., & Parkin, R. (1998). Computer-aided treatments of mental health problems. Clinical Psychology: Science and Practice, 5, 151-170.

Newman, M. G., Consoli, A., & Taylor, C. B. (1997). Computers in assessment and cognitive behavioral treatment of clinical disorders: Anxiety as a case in point. Behavior Therapy, 28, 211-235.

Nickelson, D. (1997, August). Telehealth poses opportunities and challenges for psychology. Practitioner Focus [Newsletter of the American Psychological Association, Public Relations and Communications Practice Directorate]. Retrieved in February 1999, from the World Wide Web: htip://www.apa.org/practice/Pf/aug97/teleheal.himl.

Speilberger, C. D., Gorsuch. R. L., & Luchene, R. E. (1970). Manual for the State-Trait Anxiety Inventory. Palo Alto, CA: Consulting Psycholo-ists Press.

Talley, J. L. (1987). Interactive Health Systems: Therapeutic Learning Program evaluation study. Unpublished manuscript, Interactive Health Systems, Santa Monica, CA.

Welsh, G. S. (1956). Factored dimensions A and R. In G. S. Welsh & W. G. Dahlstrom (Eds.), Basic readings on the MMPI in psychology and medicine. Minneapolis. MN: University of Minnesota Press.

5