What is an evidence hierarchy?
  • Register

Evidence hierarchy: What is it?

An evidence hierarchy classifies evidence in terms of its ability to detect effectiveness

Although a large body of research exists in the field of education, it is not all useful for every purpose and it is important to distinguish research that provides a strong indication of the direct impact of a program from research that is less able to do. Research that is more useful in determining effectiveness emphasises the ability to show that results were a direct outcome of the program being evaluated. When the evidence provides stronger indications of outcomes, there can be greater confidence that replicating the program will lead to similar outcomes.

Evidence is classified into three categories

Statistical terms explained

Meta-analysis

A meta-analysis is a synthesis of multiple other studies and usually refers to the process its authors undertook to contrast and combine the results from these studies. The purpose of a meta-analysis is to identify patterns that only come to light by looking at multiple study results. Provided the authors adhere to strict guidelines when choosing studies to include, a meta-analysis of multiple randomised controlled trials, for example, would be more reliable than a single randomised controlled trial.

Treatment and control groups

To determine the impact of a particular program or method, it is best to compare a group of subjects who are exposed to the program or method (the ‘treatment group’) with a group of subjects who have similar characteristics but are not exposed to the program or method (the ‘control group’). If the individuals in the treatment group later have better outcomes than the individuals in the control group, we can have some confidence that it is the program or method being tested that has caused the better outcomes, rather than some other factor.

Randomised controlled trials

Randomised controlled trials, or RCTs, are widely considered to contain the most robust research methodology. They include a treatment group and a control group and subjects are randomly allocated to one or the other group. Because subjects are allocated randomly, particular characteristics are more likely to be spread evenly between the two groups, thus minimising the chance that any one of these characteristics (which are not being studied) will influence the outcome of the study.

Quasi-experiments

Quasi-experiments share similarities with randomised controlled trials. They include a treatment group and a control group, but unlike RCTs, they do not involve random allocation of study subject to group. This means that they are more likely to have biases, as there may be some pre-existing differences between the groups that might cause a difference in outcomes. An example would be comparing the effect of a particular program on students at two different schools, one of which has a much lower student-teacher ratio. The experimenter is unable to assign subjects to treatment groups – the subjects are already in pre-existing groups.

...and more

Visit the glossary page.