Pragmatic Trials Collaborative

Although randomized controlled trials are the cornerstone of medical evidence, and the basis for all treatment guidelines, there are systematic gaps in this evidence which need to be filled.

What is a pragmatic trial?¹

Most randomized controlled trials (RCTs) are "explanatory" or "efficacy" trials, designed to answer whether a treatment can demonstrate benefit in a select population. Such trials normally compare an intervention to placebo or to another active intervention that may not be standard of care. They also focus on a narrow set of outcomes to which the intervention is targeted. In contrast, "pragmatic" or "effectiveness" trials: 1) examine interventions in a broader population representative of those who will be treated; 2) report outcomes of importance to patients that are intended to capture global benefit and harm; and 3) often compare the intervention in question to standard therapies. Compared to efficacy trials, pragmatic trials are intended to answer the questions that clinicians and patients have regarding therapy.

The need for more representative subjects

Most RCTs use narrow inclusion and exclusion criteria to select the participants most likely to benefit from an intervention and least likely to experience harm (e.g. targeting those at high cardiovascular risk but excluding those with renal insufficiency). This approach maximizes the likelihood of observing benefit. However, this practice also results in many trials excluding subjects similar to the patients most commonly encountered in clinical practice. In a systematic sampling of RCTs published in high impact journals, 38.5% of RCTs excluded older adults, 81.3% excluded individuals with common medical conditions, and 54.1% excluded individuals receiving commonly prescribed medications.²

Considering the multiple morbidities present in 71% of diabetics, 82% of osteoarthritics, 83% of chronic obstructive pulmonary disease sufferers, and 92% of those with coronary artery disease,³ such trials are clearly not representative of real world populations.

Of 20,388 US Medicare patients ≥ 65 years of age, only 1 in 5 patients discharged from an acute care hospital with a diagnosis of congestive heart failure (CHF) met the criteria for enrollment in 3 landmark trials that guide the treatment of all CHF patients.⁴ As a general rule, although older adults and patients with multiple co-morbidities are often the target of clinical practice guidelines, they are poorly represented in the evidence-generating trials upon which clinical guidelines are based. This is especially important for the frail elderly, who fall into both categories. There are observational data raising questions as to the value of lowering blood pressure and blood sugar in the frail elderly.^{5, 6} Such studies are only hypothesis generating but clinical trials in this traditionally understudied population are clearly needed.

The need for better assessment of harm

The use of highly selected and generally healthier patients in efficacy trials increases the likelihood such trials will fail to adequately predict harm in a population with a broader spectrum of disease. This was well demonstrated following publication of the Randomized Aldactone Evaluation (RALES) trial when, coincident with increasing use of spironolactone in the general population, rates of hospitalization for hyperkalemia in Ontario rose from 2.4 per 1000 patients in 1994 to 11.0 per 1000 patients in 2001.⁷

The potential for adverse drug-drug or drug-disease interactions increases exponentially with each additional co-morbidity and each additional medication.⁸ It is reasonable to question whether the risk-benefit ratio of a new medication might differ meaningfully when that drug becomes the 7th or 8th in use by a patient with 4 co-morbidities as compared to a single drug in an uncomplicated patient.

Efficacy trials at times also employ run-in periods to exclude subjects who do not tolerate the drug, thus reducing the chances of detecting poor tolerability or harm. Given that clinical trials place less emphasis on the evaluation of adverse effects than on primary outcomes⁹, and given that adverse drug reactions are so varied in presentation (often mimicking medical diseases¹⁰), harmful effects of drugs may not be detected until years after a drug is approved for use.

The need for patient-oriented outcomes that matter

Much of the evidence in support of common interventions is based on surrogate outcomes such as HbA1c. This is problematic since recent studies challenge the assumption that surrogates can reliably predict the effect an intervention will have on "hard" outcomes.¹¹ For example rosiglitazone lowers HbA1c but increases cardiovascular events.¹² Complex clinical scales come somewhat closer to measuring the patient’s experience but these, often arbitrary combinations of clinical signs and symptoms, are also problematic in that their clinical meaning is often difficult to determine. As much as possible, outcome measures should be highly objective (e.g. mortality, disabling stroke, hospitalization, nursing home admission, hip fracture) and resonate with patients and clinicians alike (e.g. falls, cognitive impairment, independence, quality of life).