When is a further clinical trial justified?

In an era where the increasing burden of disease and disability from chronic diseases such as back pain, stroke and diabetes, have major socio economic implications, current models of healthcare are not sustainable; and with Australia's national health expenditure expected to reach $246 billion by 2033, more innovative and cost effective approaches to healthcare are urgently needed. The work of Dr Manuela Ferreira explores this important question.

While looking at new ways to prevent and treat disease is one part of the equation, how this research is conducted is another. Randomised clinical trials are extremely costly, the average cost per participant is estimated at AU$3,700 and total trial costs have reached exorbitant figure such as US$60million.

Even the best and largest clinical trials do not always provide convincing evidence for the effectiveness or lack of effectiveness of a health intervention such as medical procedure or medication.

While randomised controlled trials are the most robust way to test such an intervention, consensus is usually only achieved when data from several and similar, high quality trials are pooled together demonstrating a significant effect from the intervention.

Until such a consensus is achieved, researchers may claim that a new trial is justified. Yet, conducting randomised clinical trials is costly, time-consuming and can subject patients to inconveniences, such as being allocated to a placebo intervention or being subjected to long assessments and numerous follow-ups.

So when is a further trial justified?

Two conditions key things are needed before deciding whether a further trial is warranted: if the results of existing trials do not clearly confirm nor rule out the worthwhile effects of a certain intervention; and, if there is reasonable chance a new trial will provide data which, when added to existing evidence, will change this uncertainty into clear evidence for or against the existence of a clinically important effect from an intervention.

The consideration here is no longer limited to whether an intervention is effective or not, the more conventional approach, but rather if the size of that effect is clinically worthwhile or not -that is, when an intervention does more good than harm and the size of the treatment effect, in the opinion of the consumer, is big enough to outweigh the costs, inconveniences and any potential risks from the intervention.

So how do we decide if data from existing trials confirm a worthwhile effect?

A good way to distinguish between interventions which have trivial effects and those that have clinically worthwhile effects is to determine if the intervention does more good than harm i.e. the treatment effect needs to be big enough to outweigh the costs and risks of the intervention.

This decision is usually based on an empirically estimated threshold, termed the 'smallest worthwhile effect' of an intervention. This should be based on the opinions of consumers, account for the risks, costs and inconveniences of the intervention, and be estimated in terms of receiving or not receiving the particular intervention.

The interpretation of the results from randomised clinical trials should only yield three conclusions: it is clear that the effect of the intervention is worthwhile; it is clear that the effect of the intervention is not worthwhile; or, it is unclear whether the effect of the intervention is worthwhile.

Can we change certainty in current evidence? Does size matter?

Although common belief suggests that very large trials will provide certainty about the effects of intervention, there is now research that shows that sometimes no trial of any size, when added to existing evidence, will be able to provide a high degree of certainty about whether the effects of intervention are worthwhile.

Statistical simulations can be used to demonstrate the impact of adding data from a new trial to current evidence. In this instance, a further trial should only be conducted if addition of data from the new trial to those of existing trials (i.e. in a meta-analysis) could remove uncertainty about whether the effects of an intervention are worthwhile (obviously, an important requirement that needs to be met by future trials in changing uncertainty is also the adequate control of risk of bias and intervention delivery).

Summary

More robust methods should be used before we decide whether or not a further trial needs to be conducted, and when the existing evidence suggests either that an intervention is clearly worthwhile, or clearly not worthwhile there may be no need for a further clinical trial. When it is unclear whether the intervention has worthwhile effects, new statistical methods (such as extended funnel plots), can be used to explore the potential impact of a new trial.

A health intervention produces worthwhile effects when it does more good than harm. The role of clinical trials is to ascertain whether those effects outweigh risks, costs, and inconvenience and the sample size requirements for a new trial should be based on consideration of the impact of the new trial on an updated meta-analysis of the evidence from all trials. Just as a more sustainable and innovative approach to healthcare is a priority to tackle the health challenges of 21st century, equally a priority is the careful consideration of when to conduct a further clinical trial and the implications of costs and the potential evidence it will produce.

This article is based on a paper published in BMJ.com in 2012. Click here for the full article.