Wikiversity

Evidence-based assessment/Prediction phase

< Evidence-based assessment

The first phase of assessment involves making rapid decisions about contending hypotheses, deciding which to evaluate further to build a case formulation and a treatment plan. Listing the most common disorders and benchmarking the base rates are the preamble to the process. They create a shortlist of hypotheses that will be worth considering precisely because they are commonplace. The list functions as a baseline set of hypotheses. We then look for disconfirming evidence as well as confirmatory evidence. The top panel of Figure 1 illustrates a graphical way of viewing the common issues as leading initial hypotheses that warrant assessment.

Click Here for Landing Page
Click Here for Landing Page
HGAPS New for Fall 2022: HGAPS and Psychology Conferences
Click Here for Landing Page
Click Here for Landing Page

HGAPS is finding new ways to make psychological science conferences more accessible!

Here are examples from APA 2022 and the JCCAP Future Directions Forum. Coming soon... ABCT!
~ More at HGAPS.org ~


Medical disclaimer: This page is for educational and informational purposes only and may not be construed as medical advice. The information is not intended to replace medical advice offered by physicians. Please refer to the full text of the Wikiversity medical disclaimer.



Studies of clinical decision making find that when we use unstructured interviews, we tend to formulate one hypothesis based on the presenting problem (usually in the first few minutes of the interview!) and then we do an excellent job of searching for confirmatory data.[1] We tend not to look for disconfirming evidence, and we also rarely consider competing or augmenting hypotheses.[2] These dynamics play into our tendency to underestimate comorbidity and to have “favorite” diagnoses that we identify at high rates.[3][4] The cognitive heuristics can be particularly error prone when working with minority groups, who may use different language to describe the presenting problem – leading to a different starting hypothesis. Consider the case of pediatric bipolar disorder: Black, low income parents are more likely to describe their concerns as focused on the youth’s behavior, and white middle class families are more likely to describe their main worry as mood swings.[1] One description pulls for an initial hypothesis of conduct problems, and the other for a mood disorder conceptualization. The confirmatory bias kicks in immediately, and if we do not systematically assess for potentially disconfirming information, then the black child winds up diagnosed with conduct disorder, and the equally labile white youth diagnosed with bipolar – exactly the pattern we see in services data.[5][6] In normal clinical practice, we do not receive corrective feedback[7] – there are no structured diagnostic interviews of a subset of cases, it is not common to hear contrasting formulations or contradictory opinions at case conferences, and if treatment does not progress because the initial assessment was off, there are a host of other reasons that are likely to come to mind first (e.g., family is too busy, not ready for change).[7] The benchmarks remind us that these disorders are equally common in both demographic groups and deserve equal initial consideration.[8][9]

Rationale

edit

EBM Decision Making and Zones of Clinical Action

edit

We have extended the EBM model by adding the traffic light color metaphor to label the zones, so that the region below the Wait-Test threshold is the “Green Zone,” the middle region betwixt the Wait-Test and Test-Treat thresholds is the “Yellow Zone,” and above the Test-Treat is the “Red Zone."[10][11] The color labels are easy for families to comprehend. The bars in Figure 1 are shaded to indicate into which zone they fall. Another refinement was to mash up the EBM thresholds with the community mental health idea of primary, secondary, and tertiary intervention.[12] Whereas classic EBM only thinks of treatment in the Red Zone, the community mental health model of levels of intervention encourages us to think about primary prevention options in the green zone (low cost, low risk, and likely to avert later problems).[10] Similarly, the Yellow Zone could be the place for targeted intervention for at risk groups, or the deployment of broad spectrum treatments that are unlikely to harm and that may have some efficacy while we continue intensive assessment to refine the diagnosis. In the case of mood disorder, suggesting that someone with a family history of bipolar but no current symptoms take fish oil supplements would be an example of a Green Zone recommendation.[13] Someone with mood symptoms but insufficient information to determine whether it is a unipolar or bipolar depression could be a good candidate for psychotherapy emphasizing sleep hygiene, coping skills, and CBT components – these are likely to be helpful whether the mood disorder follows a unipolar or bipolar course, and unlikely to cause harm; so they can be started while the evaluation process is ongoing.[14] Atypical antipsychotics would be an example of a Red Zone treatment that should wait in abeyance until the probably of a bipolar diagnosis is high enough to justify the attendant risks and side effects.[15]

Some of the base rates will be high enough to start in the Yellow Zone. At a typical outpatient clinic, the base rates of ADHD, disruptive behavior, anxiety, and mood will be high enough that they should routinely be considered in evaluation. The Yellow Zone issues define the targets for our routine evaluation. If we build a core battery, we should match the measures to the Yellow Zone issues. If we use a semi-structured interview, we want to make sure that the modules cover the Yellow Zone topics, as well as less common ones that could get kicked up to Yellow Zone levels of probability via screening or identification of risk factors.

Steps to put into practice

edit

The next layer of assessment consists of brief screens, key factors from developmental history, and gathering information from collateral informants’ perspectives. The screening measures can include instruments with broad content coverage, such as the Achenbach checklists or the Strengths and Difficulties Questionnaire. [16][17] These include subscales that address symptoms associated with many of the most common issues: Internalizing or emotional problems scores inform about whether anxiety or mood disorder might be present; externalizing scores scout for disruptive behavior disorders; [18] [19] and attention problems provide data related to ADHD or learning issues. [20]

Interpreting screening measure scores

edit

Overview

edit

The purpose of this subsection is to use Bayesian probability theory in order to accurately predict diagnoses, given base diagnosis rate in the region and diagnostic likelihood ratios.

Likelihood Ratios

edit

Likelihood ratios (also known as likelihood ratios in diagnostic testing) are the proportion of cases with the diagnosis scoring in a given range divided by the proportion of the cases without the diagnosis scoring in the same range. [17] The table below shows area under the curve (AUCs) and likelihood ratios for potential screening measures.

Likelihood Ratio Comments
Larger than 10, smaller than 0.10 Frequently clinically decisive
Ranging from 5 to 10, 0.20 Helpful in clinical diagnosis
Between 2.0 and 0.5 Rarely result in clinically meaningful changes of formulation
Around 1.0 Test result did not change clinical impressions at all

"LR+" refers to the change in likelihood ratio associated with a positive test score, and "LR-" is the likelihood ratio for a low score. Likelihood ratios of 1 indicate that the test result did not change impressions at all[16]. On the other hand, likelihood ratios larger than 10 or smaller than 0.10 are frequently clinically decisive, 5 or 0.20 are helpful, and between 2.0 and .5 are small enough that they rarely result in clinically meaningful changes of formulation.


Probability Nomogram

edit
 
Nomogram to update probability using likelihood ratio

Once we know the LR, the next step is to combine it with other information about the client. One way of doing this is using a probability nomogram. A nomogram uses geometry to turn the math steps of updating a probability into a "connect the dots" exercise.

Steps for researchers

edit

Tables and figures

edit

Psychometric properties of common screening instruments

edit
Measure Format (Reporter) Age Range Administration/

Completion Time

Coverage Interrater Reliability Test-Retest Reliability Construct Validity Content Validity Highly Recommended Free and Accessible Measures
Child Behavior Checklist (CBCL)

*not free

Parent-report The CBCL, YSR, and TRF each has all of the following subscales:
  • anxious/depressed
  • withdrawn/depressed
  • somatic complaints
  • social problems
  • thought problems
  • attention problems
  • rule-breaking behavior
  • aggressive behavior
  • affective problems,
  • anxiety problems
  • somatic problems
  • ADHD
  • oppositional defiant problems
  • conduct problems
  • Higher order: internalizing and externalizing

The CBCL and YSR are have competence scales for:

  • activities
  • social relations
  • school
  • total competence[21]
 G-E *not free*

Website to purchase

Youth Self Report (YSR)

*not free

Self-report 11-18 years old[21] 10 minutes[21] *not free*

Website to purchase

Teacher’s Report Form (TRF)

*not free

Teacher-report *not free*

Website to purchase

Likelihood ratios and AUCs of common screening instruments

edit
Screening Measure (Primary Reference) AUC LR+ Score LR- Score Clinical generalizeability Study description
Child Behavior Checklist (CBCL) - Attention and Aggression Problems T-Score[22] Boys: .86 (N=111) 10.2 (>55) 0.41 (<55) Somewhat High Utilized sample ages 6-18 which consisted of 219 brothers and sisters of children who were referred to a hospital pediatric unit for ADHD or other symptoms. Half of these siblings had brothers and sisters who had ADHD, half did not.[23]
11.2 (>55)0.35 (<55)
Teacher Response Form (TRF) - Attention Problems T-Score[22]Not reported (N=184)3.66 (>70)0.73 (<70)Somewhat HighUtilized sample ages 5-12 years referred to a research clinic for assessment of ADHD. 108 children were ultimately diagnosed with ADHD, while 76 were not. LR's are calculated for discriminating between those two groups.[24]
Teacher Response Form (TRF) - Attention and Aggression Problems T-Score[22] Not reported (N=184) 4.33 (>70) 0.89 (<70)Somewhat High
Child Behavior Checklist (CBCL) - Attention Problems T-Score[25] .84 (N=187) 6.92 (>55) 0.19 (<55) Somewhat High Utilized sample ages 6-18 recruited from local pediatricians, psychiatrists, and community advertisements. Included 95 children who met criteria for ADHD. 70 of these children also met criteria for ODD/CD.[26]
12.2 (>60) 0.41 (<60)
47 (>65) 0.53 (<65)
34 (>70) 0.66 (<70)
Child Behavior Checklist (CBCL) Anxious/Depressed Scale T-score[27] .70 (N=470)[28] 3.78 (60+)[29] .39 (<60)[30] High. Large diverse sample with mixed depression sample compared to samples without depression.
CBCL Anxious/Depressed Scale T-score[31] .75 (N=1445)[32] 1.49 (raw score 9+)[32] .67 (raw score ≤)[32]
CBCL Affective Problems Scale T-score[31] .78 (N=1445)[32] 1.49 (raw score 9+)[32] .67 (raw score ≤)[32]
Youth Self Report (YSR)[31] .81 (N=207)[33] -- --
WHO-Five Well-being Index (WHO-5) 3[34] .885 (N=294)[35] 4.40 (raw score 11+)[35] .15 (raw score ≤)[35] General sample of adolescents from Norway and Denmark

Note: “LR+” refers to the change in likelihood ratio associated with a positive test score, and “LR-” is the likelihood ratio for a low score. Likelihood ratios of 1 indicate that the test result did not change impressions at all. LRs larger than 10 or smaller than .10 are frequently clinically decisive; 5 or .20 are helpful, and between 2.0 and .5 are small enough that they rarely result in clinically meaningful changes of formulation (Sackett et al., 2000).

References

edit
  1. ↑ 1.0 1.1 Carpenter-Song, E. (2009). Caught in the psychiatric net: meanings and experiences of ADHD, pediatric bipolar disorder and mental health treatment among a diverse group of families in the United States. Cult Med Psychiatry, 33(1), 61-85. doi: 10.1007/s11013-008-9120-4 Croskerry, P. (2003). The importance of cognitive errors in diagnosis and strategies to minimize them. Academic Medicine, 78(8), 775-780. doi: 10.1097/00001888-200308000-00003
  2. ↑ Garb, H. N. (1998). Studying the clinician: Judgment research and psychological assessment. Washington, DC: American Psychological Association.
  3. ↑ Dubicka, B., Carlson, G. A., Vail, A., & Harrington, R. (2008). Prepubertal mania: Diagnostic differences between US and UK clinicians. European Child & Adolescent Psychiatry, 17, 153-161. doi: 10.1007/s00787-007-0649-5
  4. ↑ Jenkins, M. M., & Youngstrom, E. A. (2016). A randomized controlled trial of cognitive debiasing improves assessment and treatment selection for pediatric bipolar disorder. Journal of Consulting & Clinical Psychology. 
  5. ↑ Arnold, L. M., Strakowski, S. M., Schwiers, M. L., Amicone, J., Fleck, D. E., Corey, K. B., & Farrow, J. E. (2004). Sex, ethnicity, and antipsychotic medication use in patients with psychosis. Schizophrenia Research, 66(2-3), 169-175.
  6. ↑ DelBello, M. P., Lopez-Larson, M. P., Soutullo, C. A., & Strakowski, S. M. (2001). Effects of race on psychiatric diagnosis of hospitalized adolescents: A retrospective chart review. Journal of Child and Adolescent Psychopharmacology, 11(1), 95-103. 
  7. ↑ 7.0 7.1 Meehl, P. (1973). Why I do not attend case conferences. In P. Meehl (Ed.), Psychodiagnosis: Selected papers (pp. 225-302). New York: Norton. Meehl, P. E. (1954). Clinical versus statistical prediction: A theoretical analysis and a review of the evidence. Minneapolis, MN: University of Minnesota Press.
  8. ↑ Alegria, M., Vallas, M., & Pumariega, A. J. (2010). Racial and ethnic disparities in pediatric mental health. Child and Adolescent Psychiatric Clinics of North America, 19(4), 759-774. doi: 10.1016/j.chc.2010.07.001
  9. ↑ Merikangas, K. R., Akiskal, H. S., Angst, J., Greenberg, P. E., Hirschfeld, R. M. A., Petukhova, M., & Kessler, R. C. (2007). Lifetime and 12-month prevalence of bipolar spectrum disorder in the National Comorbidity Survey Replication. Archives of General Psychiatry, 64(5), 543-552. doi: 10.1001/archpsyc.64.5.543
  10. ↑ 10.0 10.1 Youngstrom, Eric A. (2013-01-01). "Future Directions in Psychological Assessment: Combining Evidence-Based Medicine Innovations with Psychology's Historical Strengths to Enhance Utility". Journal of Clinical Child & Adolescent Psychology 42 (1): 139–159. doi:10.1080/15374416.2012.736358. ISSN 1537-4416. PMID 23153181. http://dx.doi.org/10.1080/15374416.2012.736358. 
  11. ↑ Youngstrom, E. A.; Choukas-Bradley, S.; Calhoun, C. D.; Jensen-Doss, A. (2014). "Clinical guide to the Evidence-Based Assessment approach to diagnosis and treatment". Cognitive and Behavioral Practice 22: 20-35. doi:doi: 10.1016/j.cbpra.2013.12.005. 
  12. ↑ Mechanic, D. (1989). Mental health and social policy. Englewood Cliffs, NJ: Prentice-Hall. 
  13. ↑ Fristad, Mary A.; Young, Andrea S.; Vesco, Anthony T.; Nader, Elias S.; Healy, K. Zachary; Gardner, William; Wolfson, Hannah L.; Arnold, L. Eugene (2015-12-01). "A Randomized Controlled Trial of Individual Family Psychoeducational Psychotherapy and Omega-3 Fatty Acids in Youth with Subsyndromal Bipolar Disorder". Journal of Child and Adolescent Psychopharmacology 25 (10): 764–774. doi:10.1089/cap.2015.0132. ISSN 1044-5463. PMID 26682997. PMC PMC4691654. http://online.liebertpub.com/doi/10.1089/cap.2015.0132. 
  14. ↑ Fristad, Mary A.; MacPherson, Heather A. (2014-05-01). "Evidence-Based Psychosocial Treatments for Child and Adolescent Bipolar Spectrum Disorders". Journal of Clinical Child & Adolescent Psychology 43 (3): 339–355. doi:10.1080/15374416.2013.822309. ISSN 1537-4416. PMID 23927375. PMC PMC3844106. http://dx.doi.org/10.1080/15374416.2013.822309. 
  15. ↑ McClellan, Jon; Kowatch, Robert; Findling, Robert L.. "Practice Parameter for the Assessment and Treatment of Children and Adolescents With Bipolar Disorder". Journal of the American Academy of Child & Adolescent Psychiatry 46 (1): 107–125. doi:10.1097/01.chi.0000242240.69678.c4. http://linkinghub.elsevier.com/retrieve/pii/S0890856709619687. 
  16. ↑ Achenbach & Rescorla. Manual for the ASEBA School-Age Forms & Profiles. Burlington, VT. 
  17. ↑ Goodman, R.; Ford, T.; Simmons, H.; Gatward, R.; Meltzer, H. (2003-01-01). "Using the Strengths and Difficulties Questionnaire (SDQ) to screen for child psychiatric disorders in a community sample". International Review of Psychiatry 15 (1-2): 166–172. doi:10.1080/0954026021000046128. ISSN 0954-0261. PMID 12745328. http://dx.doi.org/10.1080/0954026021000046128. 
  18. ↑ Meter, Anna Van; Youngstrom, Eric; Youngstrom, Jennifer Kogos; Ollendick, Thomas; Demeter, Christine; Findling, Robert L. (2014-07-01). "Clinical Decision Making About Child and Adolescent Anxiety Disorders Using the Achenbach System of Empirically Based Assessment". Journal of Clinical Child & Adolescent Psychology 43 (4): 552–565. doi:10.1080/15374416.2014.883930. ISSN 1537-4416. PMID 24697608. PMC PMC4101065. http://dx.doi.org/10.1080/15374416.2014.883930. 
  19. ↑ Ferdinand, Robert F.. "Validity of the CBCL/YSR DSM-IV scales Anxiety Problems and Affective Problems". Journal of Anxiety Disorders 22 (1): 126–134. doi:10.1016/j.janxdis.2007.01.008. http://linkinghub.elsevier.com/retrieve/pii/S0887618507000266. 
  20. ↑ William E. Pelham, Jr.; Fabiano, Gregory A.; Massetti, Greta M. (2005-08-01). "Evidence-Based Assessment of Attention Deficit Hyperactivity Disorder in Children and Adolescents". Journal of Clinical Child & Adolescent Psychology 34 (3): 449–476. doi:10.1207/s15374424jccp3403_5. ISSN 1537-4416. PMID 16026214. http://dx.doi.org/10.1207/s15374424jccp3403_5. 
  21. ↑ 21.0 21.1 21.2 "Youth Self-Report (YSR) - Portico". www.porticonetwork.ca. Retrieved 2018-06-26.
  22. ↑ 22.0 22.1 22.2 Cite error: Invalid <ref> tag; no text was provided for refs named Achenbach-1991
  23. ↑ Chen, Wei J.; Faraone, Stephen V.; Biederman, Joseph; Tsuang, Ming T. (October 1994). "Diagnostic accuracy of the Child Behavior Checklist scales for attention-deficit hyperactivity disorder: A receiver-operating characteristic analysis". Journal of Consulting and Clinical Psychology 62 (5): 1017-25. doi:10.1037/0022-006X.62.5.1017. PMID 7806710. 
  24. ↑ Tripp, Gail; Schaughency, Elizabeth A.; Clarke, Bronwyn (2006). "Parent and teacher rating scales in the evaluation of attention-deficit hyperactivity disorder: Contribution to diagnosis and differential diagnosis in clinically referred children". Journal of Developmental and Behavioral Pediatrics 27 (3): 209-18. PMID 16775518. 
  25. ↑ Achenbach, Thomas M. (1991). Child behavior checklist for ages 4-18. Burlington, VT: Department of Psychiatry, University of Vermont. ISBN 978-0-938565-08-6. 
  26. ↑ Hudziak, James J.; Copeland, William; Stanger, Catherine; Wadsworth, Martha (October 2004). "Screening for DSM-IV externalizing disorders with the Child Behavior Checklist: A receiver-operating characteristic analysis". Journal of Child Psycholology and Psychiatry 45 (7): 1299-307. doi:10.1111/j.1469-7610.2004.00314.x. PMID 15335349. 
  27. ↑ Achenbach, Thomas M. (1991). Child behavior checklist for ages 4-18 ([11th print.] ed.). Burlington, VT: T.M. Achenbach. ISBN 0938565087. 
  28. ↑ Nolan, EE; Sverd, J; Gadow, KD; Sprafkin, J; Ezor, SN (December 1996). "Associated psychopathology in children with both ADHD and chronic tic disorder.". Journal of the American Academy of Child and Adolescent Psychiatry 35 (12): 1622-30. PMID 8973069. 
  29. ↑ Nolan, EE; Sverd, J; Gadow, KD; Sprafkin, J; Ezor, SN (December 1996). "Associated psychopathology in children with both ADHD and chronic tic disorder.". Journal of the American Academy of Child and Adolescent Psychiatry 35 (12): 1622-30. PMID 8973069. 
  30. ↑ Nolan, EE; Sverd, J; Gadow, KD; Sprafkin, J; Ezor, SN (December 1996). "Associated psychopathology in children with both ADHD and chronic tic disorder.". Journal of the American Academy of Child and Adolescent Psychiatry 35 (12): 1622-30. PMID 8973069. 
  31. ↑ 31.0 31.1 31.2 Achenbach, Thomas M. (1991). Child behavior checklist for ages 4-18 ([11th print.] ed.). Burlington, VT: T.M. Achenbach. ISBN 0938565087. 
  32. ↑ 32.0 32.1 32.2 32.3 32.4 32.5 Eimecke, SD; Remschmidt, H; Mattejat, F (March 2011). "Utility of the Child Behavior Checklist in screening depressive disorders within clinical samples.". Journal of affective disorders 129 (1-3): 191-7. PMID 20825996. 
  33. ↑ Rey, JM; Schrader, E; Morris-Yates, A (September 1992). "Parent-child agreement on children's behaviours reported by the Child Behaviour Checklist (CBCL).". Journal of adolescence 15 (3): 219-30. PMID 1447409. 
  34. ↑ Bech, P; Olsen, LR; Kjoller, M; Rasmussen, NK (2003). "Measuring well-being rather than the absence of distress symptoms: a comparison of the SF-36 Mental Health subscale and the WHO-Five Well-Being Scale.". International journal of methods in psychiatric research 12 (2): 85-91. PMID 12830302. 
  35. ↑ 35.0 35.1 35.2 Christensen, KS; Haugen, W; Sirpal, MK; Haavet, OR (June 2015). "Diagnosis of depressed young people--criterion validity of WHO-5 and HSCL-6 in Denmark and Norway.". Family practice 32 (3): 359-63. PMID 25800246. 
Retrieved from "https://en.wikiversity.org/w/index.php?title=Evidence-based_assessment/Prediction_phase&oldid=2287901"