At a time when the entire world is undergoing the largest vaccination program ever rolled out by modern medicine, it has never been more important to understand how drug trials actually work.
In general, trials of new drugs or treatments boil down to a simple analysis of risk versus benefit — do the benefits of taking a certain drug outweigh the risks?
Finding out is a methodical business, which goes something like this.
Before any new drug can be marketed, it has to be approved as being safe and effective by the relevant medicines authority in any territory in which it might be sold — such as the Food and Drug Administration in the US, the Medicines and Healthcare products Regulatory Agency in the UK, the European Medicines Agency in Europe and the Ministry of Health in the UAE.
To win approval, any new drug or treatment must first go through a series of trials. As no more than two new compounds in 10,000 end up on pharmacists’ shelves after a process that can last between 10 and 15 years, this represents a major, expensive gamble for any drug company.
First, a proposal to run a trial must be approved by the regulatory authority. Every detail of the trial design must be included in the proposal, including how the results will be analyzed, ethical considerations and at what point the trial should be halted if participants are suffering unexpected and potentially dangerous side effects.
Drugs trials are divided into three phases. In Phase I, the drug is given to a small number of healthy volunteers, chiefly to see if it is safe and has no unacceptable side effects.
(Tip: hesitate to volunteer for a Phase I trial. Disasters are rare, but they do happen. In 2006 six men who took part of a Phase I trial in a London hospital of a new anti-inflammatory drug designed to treat conditions including rheumatoid arthritis and leukaemia suffered multiple organ failure. They survived, but on balance might not have considered the £150 payment they received as worth it.)
If a drug is proved safe in Phase I (and the vast majority are), Phase II trials begin. This time, the objective is to see if the drug works as intended in people who have the condition in question.
Larger numbers of volunteers
are needed for Phase II trials, partly because they must include people with a wide range of ages and, often, ethnicities. Also, to see if the drug is any better than an existing treatment, or no treatment at all, there must be sufficient numbers of volunteers to allow them to be divided into two statistically relevant groups, or cohorts: one cohort will be given the new drug, and the other won’t.
This is where we meet the Randomized Controlled Trial, regarded as the gold standard test of a new drug.
“Controlled” means the effectiveness of the drug is compared with either an existing treatment, or a dummy treatment (known as a placebo, which is a drug or tablet that contains no active ingredient).
“Randomized” means the volunteers will be randomly allocated to receive either the drug under test or the “control” placebo or treatment.
And, to remove any chance of bias in the results, such trials are usually “double blind” — meaning neither the researchers nor the volunteers know who has been given what.
Many trials fall at this hurdle, usually because they fail to demonstrate effectiveness, but those that pass this test proceed next to Phase III.
Phase III is much like Phase II, only on a much larger scale — hundreds of volunteers, usually across multiple centers and perhaps even different countries, and often lasting years.
The rapidly developed COVID vaccines were a remarkable exception. In the face of the unprecedented global emergency, and with the cooperation of governments and regulatory authorities worldwide, the development process was accelerated dramatically. It also helped that researchers were able to take advantage of decades of previous research and development of vaccines for related viruses.
The speed at which the COVID-19 vaccines have been developed has, of course, given ammunition to vaccine skeptics, who say there is no information about the long-term effects of these drugs. This is true. But dramatic side effects are almost always experienced within a very short time of a drug being administered, and any longer-term effects will be picked up by ongoing monitoring of large cohorts of people who have had the vaccines.
This is where the all-important business of risk-benefit analysis comes into play. Does the drug cause more harm than good — or are existing treatments just as good, or better?
In trials of all the new vaccines, the risk of dying from COVID-19 was judged as being higher than the risk of dying from a side effect of the drugs.
And what the trials lacked in longevity, they more than made up for in scale, delivering some of the biggest Phase III trials in history; Johnson & Johnson enrolled 60,000 people in its trial of Ad26.COV2-S, a single-dose vaccine against Covid-19; 30,000 took part in trials for AstraZeneca’s AZD1222 and 44,000 in Pfizer/Biontech’s BNT162b2 Phase III trial.
Setting up and carrying out such trials is one thing; analyzing them statistically introduces a whole new level of complexity to the process.
Most trials set out to test a drug
against several primary, or most important, outcomes. Typically, does the drug cure a condition, make it worse, or increase or reduce the risk of death? Secondary outcomes and side-effects can also be studied: does the drug make people sick, cause headaches, heart palpitations?
For example, a recent Phase II trial of the drug Infliximab for the treatment of COVID-19, in which 17 patients in critical respiratory failure with COVID-19 were all given the drug, had two primary outcomes: the time to improvement of a patient’s oxygen levels, and the proportions of patients whose oxygen levels improved.
There were also 14 secondary outcomes. These included survival (the number of patients still alive 28 days from the start of the study), duration of hospitalization and the number of patients who required mechanical ventilation.
Once all the data have been collected, it is analyzed according to tried and tested statistical methods. The results are then written up in one or more papers that are peer-previewed (examined critically by experts not involved in the research) before they are accepted — or not — for publication in medical journals.
In the past, some drug companies have failed to publish the results of trials in which their products have performed poorly. This practice has been clamped down on since 2015, when the World Health Organisation insisted that the results of all trials, good or bad, new or old, should be made available on a publicly accessible register.
Only now can the drug be submitted for approval to national regulatory agencies, a process that involves the submission of all the files associated with the research and which, in normal times, can take years to complete before approval is given and the drug can be mass-produced and marketed.
One other important – and confusing – factor to consider when discussing drug trials is the concept of risk.
Vaccine skeptics, for example, will argue that COVID-19 vaccines are far less effective than stated. Instead of the 95 percent percent protection claimed by Pfizer, for example, they claim their real risk reduction is is only 0.84 percent.
This is based on the difference between two measurements: relative risk reduction (RRR) and absolute risk reduction (ARR).
RRR is the difference in outcome between two groups – in this case, the jabbed and the unjabbed. So if, for example, one in every 10,000 unvaccinated people contracts COVID-19, compared with only one in every 10 million vaccinated people, then the RRR is 90 percent.
Absolute risk reduction is a little more complicated and a much less impressive number, and also less-rarely cited — and understood. But it does help to give context to clinical trial findings. It refers to the statistical probability that something will happen.
So if the ARR is .84 percent, for example, that means out of 100 people treated with a drug, .84 percent would be prevented from developing bad outcomes.