Evidence-based assessment/Step 1: Plan for most common issues in clinic setting

Click Here for Landing Page
Click Here for Landing Page
HGAPS New for Fall 2022: HGAPS and Psychology Conferences
Click Here for Landing Page
Click Here for Landing Page

HGAPS is finding new ways to make psychological science conferences more accessible!

Here are examples from APA 2022 and the JCCAP Future Directions Forum. Coming soon... ABCT!
~ More at HGAPS.org ~

Medical disclaimer: This page is for educational and informational purposes only and may not be construed as medical advice. The information is not intended to replace medical advice offered by physicians. Please refer to the full text of the Wikiversity medical disclaimer.

Overview edit

Pareto's 80:20 Rule of Thumb (or "Law of the Vital Few") provides a guiding example for implementing EBA. There can be an incredible richness and complexity to people and their challenges and circumstances. If we try to deal with all the complexity at the beginning, we will not get very far with the limited amount of time and resources that we have. Instead, focusing on the most common issues first will give us things that we can use for most (if not all) of the cases with whom we work. Taking care of these topics first maximizes the return on investment in getting our practice organized.

The first step is to gather descriptive information and external benchmarks about what are common issues in clinical settings similar to ours.

We can also consider a formal audit of our local practice and rates, and make a list of what are the most common presenting problems and needs.

We then want to start gathering resources, including assessment tools that address the different issues. If we are working at an established clinic, then we can sort the measures that we have and tag them to use for specific assessment questions. At this stage, we are looking for gaps, where we may have a frequent issue that does not have a matching assessment tool. If we have a gap like this, then it becomes a top priority to find a reasonable tool for filling the gap. If all the bases are covered, then we can start evaluating whether there are "upgrades" available that would provide better utility for our clients.

Rationale edit

Focusing on the most common issues is a strategy for trying to accomplish the most coverage with an initial investment of time and effort.

One interesting question is, "What counts as an issue?" DSM diagnoses, clinical issues that are not a diagnosis (couples in distress; parent child conflict), transdiagnostic issues (e.g., NSSI)....

A second issue is, "What definition do we use?" This includes issues of local diagnoses and base rates, versus external benchmarks such as epidemiological studies and meta-analyses. Discuss billing diagnoses versus actual diagnoses; unstructured versus semi-structured or structured approaches.

A third issue is type of setting -- the community epidemiological samples include lots of people who are not seeking treatment. This leads to interesting points about Berkson's bias, differences in stigma (and culture as a component affecting this)....

We talked about epidemiological studies at some length -- could be interesting and helpful to add links to some of the major ones (like NCS-R, NCS-A, NESARC, NLAS, NHANES...). We also talked about diagnosis as usual (DAU) versus semi/structured diagnostic interviews. Interesting points about DAU:

  • It reveals local practices and norms (attachment example -- is reactive attachment disorder "real" or "fluffy" depends a lot on where you are working)
  • It helps identify topics where you may want to upgrade your reading or skills quickly if you are changing settings (as happens frequently during graduate training, internship, postdoctoral placements). When we identify a priority, look for reviews focused on that topic (and read the chapter, not the whole book!).
  • Discrepancies are informative (see "Benchmarking" -- next section)
  • These chart diagnoses have low inter-rater reliability, and they can have systematically biased validity due to billing and reimbursement issues (e.g., conduct disorder example).
    • This has implications for the machine-learning approaches that people are starting to use with big data. If the main concern were low reliability, then big enough samples could compensate for that in theory. But if the diagnoses used to train the machine learning model are systematically biased, that creates major problems for the model. Machine learning will have a hard time using billing diagnoses to "learn" about predictors of conduct disorder, because (a) although conduct disorder exists, and there will likely be meaningful predictors and correlates available to the model, (b) conduct disorder is not diagnosed accurately (if at all) in billing diagnoses.

Still to discuss: Nuances of CDC surveillance data; and we did a shout out to LCA/GMM/Cluster analysis as multivariate methods that could define groups (potentially replacing DSM diagnoses and single variable ROC approaches).

Steps to put into practice edit

There are several different options for approaching the "common issues."

One is to look at what experts recommend. Handbooks often build their table of contents around what are seen as the most common issues. The web sites curated by different professional societies also provide good coverage of common issues.

A second strategy would be to gather estimates from external sources. We have pulled together several options on this page and on the "Benchmarking" pages that come in the next section.

A third option is to do an audit or build a local list of main issues. This could be as simple as jotting down a list of "Top Ten Clinical Issues," or it could be a formal query of a database of electronic medical records. At UNC Chapel Hill, we have been using a program called Titanium for our electronic medical record for several years. We pulled a report of the diagnoses that we entered as part of our routine clinical practice, and we will show some examples of the steps involved in using this to estimate local base rates.

Expert Recommendations edit

For now, we will focus first on clinical psychology, and loosely divide things into "youth" and "adult" assessment. Over time, we will expand to add other age groups, specialized populations, and additional professions.

Child and Adolescent Assessment Recommendations edit

Professional Society Web Sites edit
Handbooks edit
  • Assessment in Children and Adolescents (4th ed.). New York, NY: Guilford Press.
    • [Disclaimer: Eric Youngstrom and Mitch Prinstein are going to be the editors for the 5th edition of this book, which is tentatively scheduled for publication in late 2018 or 2019]
  • Dulcan

Adult Assessment Recommendations edit

Professional Society Web Sites edit

Division 12

[American Counseling Association]

Handbooks edit

A guide to assessments that work. New York, NY: Oxford University Press.[1]

Local Rates edit

First, we should acknowledge that a lot of experts think that using local rates is a bad idea. We'll briefly go through the biggest concerns.

External Benchmarks edit

We discuss these in more detail in the next section.

Annotated bibliography edit

Tables and figures edit

Prevalence benchmarks for clinical issues discussed

(Rettew et al., 2009)[2]

Condition A Guide to Assessments That Works Chapter[3] CDC DAU SDI General Population*
ADHD 4 11% or 6.8%[4] 23% 38%
Conduct Problems 5 3.5%[4] 17% CD, 37% ODD 25% CD, 38% ODD 1.5-3.2% antisocial[5]
Mood Disorders 17% MDD, 10% dysthymia 26% MDD, 8% dysthymia
  • Pediatric Depression
6 2.1%[4] -- --
  • Adult Depression
7 -- -- --
  • Late Life Depression
8 -- -- --
  • Bipolar
9 -- -- -- 2.9%NCS-A[6], 1% bipolar I, 2-4% spectrum [7]
Self-Injurious Thoughts and Behaviors 10
  • NSSI
  • Ideation
  • Attempt
Anxiety 3.0%[4]
  • Child and Adolescent
11 8% 18%
  • Specific and Social Phobia
12 --- 6% (social) 20% (social) 9% social & 19% specificNCS-A[6]
  • Panic and Agoraphobia
13 -- 12% (panic) 11% (panic) 2.4% agora/2.3%panicNCS-A[6]
  • Generalized Anxiety Disorder
14 -- 5% 10% 2.2%NCS-A[6]
  • Obsessive-Compulsive Disorder
15 -- 9% 12% 1-2%(Ch. 15)[8]
  • Post-Traumatic Stress Disorder
16 -- 3% 9% 5%NCS-A[6], 3.6%NCS18-54
Substance Use Disorders 17 4.7%[4] 14% 17% 7.4̤̥%[9]
Alcohol Use Disorder 18 4.2%[4] 10% 13%
Gambling Disorders 19 -- -- --
Schizophrenia 20 -- -- -- 0.014% child [10]
Personality Disorders
  • Cluster A
  • Cluster B (antisocial separate)
  • Cluster C
21 -- -- --
  • (Cluster A) 3-6% life[5]
  • (Cluster B) 1-5%% life[5]
  • (Cluster C) 2-4% life[5]
Couple Distress 22 -- -- -- ~50% divorce rate(Ch. 22)[11]
Sexual Dysfunction 23 -- -- --
Eating Disorders 24 -- -- --
Sleep/Wake Disorders 25 -- -- --
Pain in Children and Adolescents 26 -- -- --
Chronic Pain in Adults 27 -- -- --

References edit

  1. Hunsley, J., & Mash, E. J. (2008). Oxford Textbooks in Clinical Psychology : Guide to Assessments That Work. Cary, GB: Oxford University Press, USA. Retrieved from http://www.ebrary.com
  2. RETTEW, D. C., LYNCH, A. D., ACHENBACH, T. M., DUMENCI, L., & IVANOVA, M. Y. (2009). Meta-analyses of agreement between diagnoses made from clinical evaluations and standardized diagnostic interviews. International Journal Of Methods In Psychiatric Research, 18(3), 169-184. doi:10.1002/mpr.289
  3. Hunsley, John; Mash, Eric J. (2008). A guide to assessments that work. New York: Oxford University Press. ISBN 0198042450. OCLC 314222270. https://www.worldcat.org/oclc/314222270. 
  4. 4.0 4.1 4.2 4.3 4.4 4.5 "Mental Health Surveillance Among Children — United States, 2005–2011". www.cdc.gov. Retrieved 2017-03-09.
  5. 5.0 5.1 5.2 5.3 Fonagy, P., Roth, A., & Higgitt, A. (2005). Psychodynamic psychotherapies: Evidence-based practice and clinical wisdom. Bulletin of the Menninger Clinic, 69(1), 1-58. Retrieved from http://libproxy.lib.unc.edu/login?url=http://search.proquest.com/docview/200493862?accountid=14244
  6. 6.0 6.1 6.2 6.3 6.4 Merikangas, Kathleen Ries; He, Jian-ping; Burstein, Marcy; Swanson, Sonja A.; Avenevoli, Shelli; Cui, Lihong; Benjet, Corina; Georgiades, Katholiki; Swendsen, Joel. "Lifetime Prevalence of Mental Disorders in U.S. Adolescents: Results from the National Comorbidity Survey Replication–Adolescent Supplement (NCS-A)". Journal of the American Academy of Child & Adolescent Psychiatry49 (10): 980–989. doi:10.1016/j.jaac.2010.05.017. PMC PMC2946114 Check |pmc= value (help). PMID 20855043.
  7. Van Meter, Anna R.; Moreira, Ana Lúcia R.; Youngstrom, Eric A. "Meta-Analysis of Epidemiologic Studies of Pediatric Bipolar Disorder". The Journal of Clinical Psychiatry72 (09): 1250–1256. doi:10.4088/jcp.10m06290.
  8. Abramowitz, Jon S. (2008). "Obsessive-Compulsive Disorder". In Hunsley, John; Mash, Eric. A Guide to Assessments that Work. New York, NY: Oxford Press. pp. 275-292. ISBN 978-0-19-531064-1.
  9. Substance Abuse and Mental Health Services Administration. (2019). "Key Substance Use and Mental Health Indicators in the United States: Results from the 2018 National Survey on Drug Use and Health". Center for Behavioral Health Statistics and Quality, Substance Abuse and Mental Health Services Administration.
  10. Thomsen, P. H. (1996-09-01). "Schizophrenia with childhood and adolescent onset — a nationwide register-based study". Acta Psychiatrica Scandinavica94 (3): 187–193. doi:10.1111/j.1600-0447.1996.tb09847.x. ISSN 1600-0447.
  11.  Snyder, D.K.; Heyman, R.E.; Haynes, S.N. (2008). "Couple Distress". In Hunsley, John; Mash, Eric. A Guide to Assessments that Work. New York, NY: Oxford Press. pp. 439-463. ISBN 978-0-19-531064-1.
  12. Youngstrom, Eric A.; Van Meter, Anna (2016-12-01). "Empirically Supported Assessment of Children and Adolescents". Clinical Psychology: Science and Practice 23 (4): 327–347. doi:10.1111/cpsp.12172. ISSN 1468-2850. http://onlinelibrary.wiley.com/doi/10.1111/cpsp.12172/abstract.