If any particular form of evidence “counts” as valid in a specific domain of practice or knowing, it is likely to have an aura of self-evidence about it. But what happens when an accepted form of evidence comes under challenge, and is no longer taken for granted or treated as standing in an unquestionably “evidential” relation to its object? Such moments have the potential to cast light on important assumptions and values that vouchsafe established forms of knowing and doing, which may otherwise be implicit and possibly resistant to critical scrutiny.
Such a challenge is underway within healthcare in relation to one established method of assessing the safety and efficacy of medicines and other pharmaceutical products: the Randomized Controlled Trial (RCT). Long regarded as the “gold standard” of evidence in medicine, RCTs are currently under challenge on multiple grounds: as being too slow and costly; as not serving the needs of patients with serious conditions for whom no effective therapies may be available; and as being ill-suited for assessing the value and benefits for medicinal products that have small evidence bases. These include therapies for patients with rare diseases (also known as orphan conditions), and some newer therapies intended for genomically stratified populations.
One charismatically designated alternative to the RCT is “Real World Evidence” (RWE), an evolving construct that broadly refers to evidence gathered as drugs are used “in real life, in real time” as opposed to the artificially controlled context of the RCT. RWE is being promoted in some quarters in tandem with what are known as “Adaptive Pathways” (APs), envisioned as more flexible and iterative trajectories for the development, approval, and use of pharmaceutical products. Whereas proponents of RWE and APs view them as pragmatic responses to the threat of spiraling drug costs and unacceptably long waits in getting promising new medicines to patients, critics view them as an assault on the scientific rigor of the RCT, and as an abdication of regulatory responsibility that will only benefit pharmaceutical interests and potentially harm patients.
On December 16, 2016 members of the “Limits of the Numerical” research group at CRASSH convened a one-day workshop to explore the relative merits of RCTs and RWE, and to deepen our understanding of some of the larger issues they engage. Treating these debates as a high-stakes instance wherein the limits of quantification are being actively asserted and contested in actual practice, we sought to assemble a small number of experts and academics to discuss how these developments are unfolding in their areas of work. One of our goals was to learn how to better align our work on the limits of quantitatively-based decision making with work being done on the proverbial frontlines. Organised as a day of informal discussion with short presentations, we also sought to learn how best to frame our questions and to explore possible answers in ways that will heighten their relevance to actors working further downstream.
Among the most notable features of the day was the lack of consensus among participants on a whole host of issues, ranging from the relative merits of RCTs and RWE, to the question of whether or not the very fact of the debate suggests that healthcare is on the cusp of an evidential and regulatory paradigm shift. The fact that no easy consensus or conclusive answers emerged in the course of the workshop underscored the possibility that one powerful form of evidence might be losing its self-evidence. It also indicated a few avenues of anthropological inquiry I will be pursuing going forward as a member of the “Limits of the Numerical” research group. For example, how do these debates articulate with developments in genomics, and with popular imaginaries regarding the transformative potential of personalized medicine? How might a shift toward RWE be both reflecting and lending coherence to new expectations concerning what counts as evidence, and by extension, how public knowledge is created? How might established structures of accountability be shifting in the process, and with what kinds of results? An anthropological approach to these and related questions will illuminate how the “limits of the numerical” are being asserted and contested in one situated domain of practice, and afford new insights into the active role played by regimes of evidence in conferring an aura of naturalness or givenness upon contingent arrangements and ways of knowing the world.
