Evaluating Evidence/Questions for Evaluating Evidence

The purpose of these questions is to determine how accurately and in what ways certain evidence represents reality. Determining the answers to these questions can help evaluate evidence being presented.

1. In what ways is this evidence relevant to the problem being investigated?
2. Where was this evidence originally found?
   1. Who originally discover this evidence?
   2. When and where was it discovered?
   3. Under what circumstance was it discovered?
   4. How has it been handled, stored, protected, or modified since it was originally found?
   5. What people, organizations, or agencies have handled, stored, or accessed this evidence since its original discovery?
3. Is this the best available evidence?
   1. Is this the original object, document, article, or photograph?
      1. Is the original available?
      2. If this is not the original, how might it differ from that original?
4. Is this direct objective evidence, or is this an interpretation of observations?
   1. Is the original evidence available to be examined?
   2. Who has had an opportunity to examine the original evidence?
   3. What observations, interpretations, conclusions, and report have they provided?
   4. What assumptions have they made?
   5. It what ways are the reported observations incomplete, inaccurate, or biased representations of the original evidence?
      1. What assumptions were made in the observation, examination, assessment, and reporting of the observations?
      2. What is omitted from the reports?
   6. Have expert and unbiased researchers had sufficient unfettered access to the original evidence to allow them to conduct a thorough investigation?
5. In what ways is evidence in the form of reports biased?
   1. What are the affiliations, motivations, and conflicts of interests of the people who sponsored the research, conducted, the research, reported the research, edited the reports, and selected this evidence?
   2. What factors are influencing the researchers, reporters, and promoters of this evidence?
   3. What evidence is available that supports other opinions, points of views, narratives. Or hypotheses?
   4. How are my confirmation biases distorting the evidence and my interpretation of the evidence?
6. Is the evidence tangible or is it in the form of testimony?
   1. If tangible evidence exists, can that be obtained and examined to avoid having to rely on testimony?
7. Are video, audio, or photographic records available?
   1. When, where, and why were these recorded?
   2. What views of the event or object are available?
   3. What gaps in the sequence of events were not recorded?
   4. What perspectives on the evidence were not recorded?
8. When testimony is used as evidence, how reliable is the testimony?
   1. Is the testimony from an eyewitness, or is it hearsay?
   2. Are rumors, gossip, or suggestions included in or allowed to influence the testimony?
   3. Does the witness have firsthand knowledge of the matter?
   4. Was the witness sober and of sound mind, free of chemical, emotional, mental or other distress, distortions or altered mental states at the time the observations took place and when the observations were reported?
   5. Was the witness coerced, pressured, or offered any incentive (material or otherwise) to testify?
   6. Were any responses suggested to the witness?
   7. What may have been used to lead the witness, aid the witness in recalling events, or to fill in gaps in the witness’s recollection?
   8. What time frames are relevant? When did the events originally occur? When were those events first witnessed? When were the first reports recorded?
9. Is this evidence anecdotal or representative?
   1. What is the full scope, universe, or population being studied?
   2. What fraction of that scope is represented by this evidence?
   3. Is the sample representative?
   4. What sampling errors or biases are present?
   5. Is the sample sufficiently large to support the conclusions being made?
   6. Do the conclusions overgeneralize from the evidence studied?
10. What expertise, if any, qualifies the original investigators and researchers to collect, examine, analyze, and report on the evidence?
    1. Are the investigators experts in the field being studied and reported on?
       1. What is the evidence for and against their expertise?
       2. What is nature of their expertise?
       3. Why is their expertise relevant to the problem being investigated?
    2. Is expertise sufficiently differentiated from authority, fame, or popularity?
11. Were the research methods used to obtain this evidence reliable methods?
    1. What methods were used?
    2. Who performed the work?
    3. Are they competent to perform this work?
    4. What is it that makes these methods reliable?
    5. How do the methods used compare to the best practices of scholarly research?
12. Have the reported results been replicated by other independent experts?
    1. Are the results published in a peer-reviewed scholarly journal?
    2. Do the results represent a broad consensus of experts in the field?
13. What are the journalism standards and editorial policies of the publications reporting the evidence?
    1. Are the journalism standards and editorial policies publicly available?
    2. Do the journalism standards and editorial policies uphold high standards of independence, objectivity, and verification of the information?
14. In what ways might have rhetoric influenced beliefs or decisions?
15. What alternative explanations or interpretations are fully consistent with this evidence?
    1. Is a prank or alien visitors the more likely explanation of crop circles?
16. Do the explanatory theories dive deep enough into the evidence?
    1. What alternative explanatory theories have been explored?
    2. What evidence supports this as the most likely conclusion?
17. In what ways does this evidence support your current worldview? In what ways does it challenge your worldview?
    1. In what ways does your current worldview influence your evaluation of this new evidence?
    2. In what ways is your current worldview creating confirmation bias?
       1. Does this bias favor or oppose the new evidence?
    3. How would your worldview have to change for you to accept this new evidence as being true, representative, and significant?
       1. Are you being guided primarily by your current worldview, or by the new evidence?
       2. How can you maintain your objectivity in evaluating the new evidence in the context of your current worldview?
18. Does this new evidence increase or decrease Global consilience? 

Further Reading

Students interested in learning more about questions that can be helpful in evaluating evidence may be interested in these resources.

• Weighing Evidence, Legal Services, Immigration and Refugee Board, December 31, 2003. Chapter 5 is particularly relevant.