Click Here for Landing Page

HGAPS New for Fall 2022: HGAPS and Psychology Conferences

Click Here for Landing Page

HGAPS is finding new ways to make psychological science conferences more accessible!

Here are examples from APA 2022 and the JCCAP Future Directions Forum. Coming soon... ABCT!
~ More at HGAPS.org ~

Medical disclaimer: This page is for educational and informational purposes only and may not be construed as medical advice. The information is not intended to replace medical advice offered by physicians. Please refer to the full text of the Wikiversity medical disclaimer.

← Process phase

Evidence-based assessment

Step 11: Progress and process measures →

Process: Goal Setting for Milestones and Outcomes

Overview

We want to keep our clients engaged with treatment and coming back until they achieve their goals. One of the best ways to do this is by working together to define goals that are achievable, measurable, and motivating to them. Having midterm and "final" goals, and having ways of tracking progress and sharing feedback session-by-session, all are ways of enhancing engagement, reducing drop-out ("premature termination"), and improving outcomes.

Rationale

Good assessment – combining formulation and detailed, personalized feedback--may be an effective intervention in its own right^[1]^[2]. A meta-analysis of 17 published studies^[3] found that assessment-as-intervention produced outcomes significantly better than the comparison group, with a Cohen’s d of .4, or roughly 2/3 of the assessment group having a better outcome than the average outcome for the comparison group not receiving the detailed feedback^[4]. Though the bulk of the studies were done with adults, the principle of providing detailed, personalized feedback is likely to generalize to family interventions; it also is a core element of coaching in sports as well as life skills. Regular feedback also reduces drop out, or premature termination^[5]^[6].

Steps to put into practice

Clinically significant change

Then standard rules of thumb would apply: 1.96 or greater would be 95% likely to reflect real change (i.e., occurring by chance less than 5% of the time), and 1.65 would be 90% likely. Others have suggested also reporting the 95% and 90% critical values in raw score metrics, as they would be easier for clinicians to use.^[7] Rather than observing that our client’s score on the Beck Depression Inventory decreased by 11 points, then dividing that change score by the standard error of the difference (4.8 points, per data reported in the technical manual) to get an RCI of 2.24, it is easier for a clinician to know that given the precision and stability of the BDI, changes of 8 points (rounding up from 7.9, the product of 1.65 and 4.8) are enough to be 90% confident and 10 (rounding up, not rounding off, per Jacobson’s recommendations) are 95% confident that they reflect a real pattern and not measurement error.^[8]

Convenience is another barrier. It is not realistic that busy clinicians would hunt down technical parameters and then perform calculations on a case-by-case basis in real time. For clinical significance to be feasible to measure in practice, the benchmarks and thresholds would need to be calculated ahead of time for the measures likely to be used (Step B), and ideally programmed into an Excel spreadsheet or Google Doc so that the clinician could just enter the score and get the results. Then, the baseline scores could be used to generate the targets for change. If a case started with an Externalizing score of T = 80, then the standard error of the difference score sets a change of 7 points as 90% likely to reflect real change (RCI > 1.65), or 8 points as 95% likely.^[9] Thresholds would be Back = 70, Closer = 56, and Away = 37 to 42 (depending on the age and sex of the youth). A way of explaining this to the family could be: “We definitely want our treatment to reduce her aggressive and rule-breaking behaviors. After some sessions, we should see progress if treatment is helping. The problem won’t be all gone, but if we asked you to fill this out as a progress check, and we saw the score drop by at least 7 points, we would be confident that treatment was helping. After some more sessions, if we got the score to less than 70, that would be another milestone that showed real progress. Youths her age show some of these behaviors, too, for all sorts of different reasons, and a score lower than 70 would be back in that ballpark of behavior, although still at the high end. With more sessions, we would like to get to the point where the score might drop further, below a 56. At that point, things would not be perfect, but the types of behaviors you would be seeing would be more typical of most teens, and not as similar to those who are coming to a clinic. If you work hard and things go very well, then scores below Away = 38 would indicate spectacular success, where they are radically different from where you started.”

Good assessment – combining formulation and detailed, personalized feedback--may be an effective intervention in its own right.^[1]^[2] A meta-analysis of 17 published studies found that assessment-as-intervention produced outcomes significantly better than the comparison group, with a Cohen’s d of .4, or roughly 2/3 of the assessment group having a better outcome than the average outcome for the comparison group not receiving the detailed feedback.^[3]^[4] Though the bulk of the studies were done with adults, the principle of providing detailed, personalized feedback is likely to generalize to family interventions; it also is a core element of coaching in sports as well as life skills. Regular feedback also reduces drop out, or premature termination.^[5]^[6]

Where to find copies of measures and supporting details

There are several websites that help gather copies of measures and information about them.

PROMIS. The NIH funded the development of the PROMIS outcome measures. These cover a wide range of health issues, including depression, anxiety, sleep, and others. The PROMIS measures currently are free to use as paper & pencil versions, and there is an iPad version with a licensing fee. The PROMIS measures are one of the first places where researchers are publishing MID benchmarks (e.g., Thissen et al., 2016). As of March 2017, these do not have the Jacobson & Truax type benchmarks for Away, Back, and Closer, nor are the SEs of the difference score usually reported.

Society for Clinical Psychology.

EBA on Wikipedia.

California Clearinghouse.

Tables and figures

References

↑ ^1.0 ^1.1 Bickman, Leonard (2017-02-23). "A Measurement Feedback System (MFS) Is Necessary to Improve Mental Health Outcomes". Journal of the American Academy of Child and Adolescent Psychiatry. 47 (10): 1114–1119. doi:10.1097/CHI.0b013e3181825af8. ISSN 0890-8567. PMC PMC2893344
↑ ^2.0 ^2.1 Carlier, Ingrid V. E.; Meuldijk, Denise; Van Vliet, Irene M.; Van Fenema, Esther; Van der Wee, Nic J. A.; Zitman, Frans G. (2012-02-01). "Routine outcome monitoring and feedback on physical or mental health status: evidence and theory". Journal of Evaluation in Clinical Practice. 18 (1): 104–110. doi:10.1111/j.1365-2753.2010.01543.x. ISSN 1365-2753. PMID 20846319.
↑ ^3.0 ^3.1 Poston, John M.; Hanson, William E. (2010-06-01). "Meta-analysis of psychological assessment as a therapeutic intervention". Psychological Assessment. 22 (2): 203–212. doi:10.1037/a0018679. ISSN 1939-134X. PMID 20528048.
↑ ^4.0 ^4.1 Lilienfeld, Scott O.; Garb, Howard N.; Wood, James M. (2011-12-01). "Unresolved questions concerning the effectiveness of psychological assessment as a therapeutic intervention: comment on Poston and Hanson (2010)". Psychological Assessment. 23 (4): 1047–1055; discussion 1056–1062. doi:10.1037/a0025177. ISSN 1939-134X. PMID 22122676.
↑ ^5.0 ^5.1 Lambert, Michael J.; Harmon, Cory; Slade, Karstin; Whipple, Jason L.; Hawkins, Eric J. (2005-02-01). "Providing feedback to psychotherapists on their patients' progress: clinical results and practice suggestions". Journal of Clinical Psychology. 61 (2): 165–174. doi:10.1002/jclp.20113. ISSN 0021-9762. PMID 15609358.
↑ ^6.0 ^6.1 Swift, Joshua K.; Greenberg, Roger P. (2012-08-01). "Premature discontinuation in adult psychotherapy: A meta-analysis.". Journal of Consulting and Clinical Psychology. 80 (4): 547–559. doi:10.1037/a0028226. ISSN 1939-2117.
↑ Youngstrom, E. A., & Frazier, T. W. (2013). Evidence-based strategies for the assessment of children and adolescents: Measuring prediction, prescription, and process. In D. J. Miklowitz, W. E. Craighead & L. Craighead (Eds.), Developmental psychopathology (2nd ed., pp. 36-79). New York: Wiley.
↑ Beck, A. T., & Steer, R. A. (1987). Beck depression inventory manual. San Antonio, TX: Psychological Corporation.
↑ Youngstrom, E. A., Freeman, A. J., & Jenkins, M. M. (2009). The assessment of children and adolescents with bipolar disorder. Child and Adolescent Psychiatric Clinics of North America, 18(2), 353-390. doi: 10.1016/j.chc.2008.12.002