Not All Evidence is Equal

One of the singular greatest advances in modern medicine, and the force that has defined it is the scientific method. Based on a body of rigorously designed tests that isolate and identify cause and effect relationships has resulted in the breadth and depth of medical knowledge and practice to increase remarkably over the last century. What’s more, the thorough and exhaustive nature of these experiments has meant that their conclusions can be acted on without doubt breeding a modern generation of doctor that is decisive, effective, scientifically literate and a practitioner of evidence based medicine. For students aiming to study medicine in Australia, understanding why doctors rely on evidence is an early step in thinking like a clinician.

Considering this, it might be understandable to be surprised at the slow pace at which change in medical practice occurs. Sometimes it can take up to fifteen years from a discovery being made to seeing it implemented in every day practice. The professional judgement needed to adopt new evidence carefully is related to scenarios explored in UCAT Situational Judgement, where safe and appropriate actions must be prioritised.

This is because evidence from any given experiment is not necessarily equivalent with others. Many variables come into play when determining the value of evidence. The number of participants in a trial, how randomly allocated they were, if there was a control or placebo group and whether the trial and results are reproducible. For assorted reasons including funding, availability of suitable subjects and time, experimenters are often required to compromise on one or more of these values to perform their study. Whilst still making a valuable contribution, the less well designed a study is, the more difficult it is to take it at face value. This same process of weighing assumptions, controls and conclusions is also useful in UCAT Decision Making, where candidates are asked to judge arguments and interpret information critically.

Therefore, we describe the “levels of evidence.” This is a hierarchy from 5 to 1 in order of increasing value of that evidence, or how confident we can be that the results are valid. For UCAT candidates, learning to rank evidence by reliability can also strengthen UCAT Verbal Reasoning, because many questions depend on separating supported conclusions from plausible but unsupported claims.

• Level five is the lowest. This is evidence based on expert “opinion” that hasn’t been critically scrutinised. This will often be based on experience or the expert’s assessment of cause and effect.
   
• Level four is evidence based on a series of isolated cases or inferior quality studies involving groups or cases and a control.
   
• Level three is divided into 3a and 3b with the former a superior level of evidence. 3b is evidence based on singular case control studies – a study where a case in which a possible cause and effect is identified, is compared with a control. 3a is a critical review of a collection of such studies as seen in 3b.
   
• Level two evidence in divided into a and b as well. 2b consists of evidence taken from a single cohort study involving groups of participants and inferior quality randomised control trials where groups of participants who receive an intervention are compared with groups that only think they have received the intervention, but in fact have not. 2a refers to a systematic review of cohort studies without control groups.
   
• Level one is the highest quality of evidence. This is also divided into 1a and 1b. 1b is evidence derived from individual, well designed, randomised control trials. 1a, the highest standard of evidence is derived from a systematic review of large groups of randomised control trials. 1a is also known in some situations as Cochrane Review.