The similarity between patient populations in clinical trials and patients treated in real-world practice is a long-standing issue in the clinical trials world. On the one hand, it seems like a good idea to conduct trials only in populations where you expect to see the greatest benefit, since this can reduce burdens on the study population and can increase the likelihood of showing that a new drug is efficacious. On the other hand, it also seems like a good idea to conduct trials in populations that resemble clinical practice as much as possible, since this often maximizes the generalizability and applicability of the findings. But because both of these are good ideas—and both approaches are important to improving the science of medicine—debate continues about which approach ought take priority.
The development of anti-programmed cell death protein 1 (anti-PD1) immunotherapies provide an instructive case in this respect. The anti-PD1 therapies, nivolumab and pembrolizumab, have received multiple FDA approvals to treat different forms of cancer. A 2018 analysis in JAMA Oncology by Jeremy O’Connor and colleagues showed that there has been swift uptake of these treatments in practice. However, even though these two therapies were supported by positive clinical trials, this does not guarantee that the patient populations studied in those trials closely resemble the patient populations treated with these therapies in practice. In particular, O'Connor et al. investigated whether the ages of patient populations in the pivotal nivolumab and pembrolizumab trials (i.e., the clinical trials used to support specific FDA approvals) differed significantly from the real-world patient populations who got the therapies once they were approved.
The figure below shows a heat map of O'Connor et al.'s data, contrasting the age distribution in the patient populations from the pivotal clinical trials versus the age distribution of patients treated in practice. The color saturation of each cell on the heat map corresponds to the proportion of patients in each age group for that row (where each row is a drug/tumor type combination). So darker cells mean more patients from that age group received the intervention. You can also mouse-over each cell to see more of the underlying data.
Heatmaps comparing age distribution of patient populations in pivotal clinical trials for pd-1 immunotherapies (purple) and patients treated with those therapies in practice (blue)
What does this show?
This heat map shows us immediately how there is a difference in the age distribution of patients enrolled in the trials of these immunotherapies and the ages of patients who get these treatments in practice. The patients in trials are consistently younger. This is not a problem in itself, but it does mean that the trial data is not as representative, and therefore not as pragmatic or practice-oriented, as it could be.
But looking beyond this specific data set, the heat mapping approach itself—i.e., breaking down the patient population into “cells” corresponding to categories of clinical interest—points us toward a better way of thinking about the information we are getting (and the information we want) from clinical trials.
Indeed, this is a rather simple breakdown of the population space into 3 patients subgroups across 5 indications, but the heat map method of organizing and presenting data could be scaled up to display a much richer picture of the population (e.g., including categories for co-morbidities or known risk factors). More data could also be included in each cell, such as safety and efficacy information.
But the bottom line is that what we get from trials isn’t just a point estimate for a population-level effect. Trials are already collecting the data that could inform how to treat patients corresponding to each “cell” on the heat map. And there is more we could learn by presenting the data in this way. We might even imagine a health system that could track the information and update the cells in real time, providing a living evidence map for patients and clinicians to use when making treatment decisions.
Such an approach would not necessarily improve the match between trial patients and real-world patients. But it would absolutely make it possible to know, at a glance, whether there is relevant trial data available.