A Perspective from the 'Reimagining Psychedelic Trials' Working Group
Written by Tehseen Noorani on behalf of the Reimagining Psychedelic Trials working group
with direct input from the following members: Matthew Baggott, Gillinder Bedi, Boris Heifets, Katie Hendy, Nicolas Langlitz, Celia Morgan, Emilia Sanabria & Harriet de Wit (all pictured).
Part of our Year in Review series
A decidedly interdisciplinary group of researchers began meeting in early 2022 to discuss issues pertinent to the design of psychedelic trials. Benefiting from their diversity of approaches and experiences, the Reimagining Psychedelic Trials working group has been engaging with a variety of key questions thus far, some of which are introduced below…
2022 saw a rise in interdisciplinary groups, events and conferences examining psychedelic research and scholarship. Comprised of scientists, social scientists and humanists, our own ‘Reimagining Psychedelic Trials’ working group began meeting early in the year. Trial design is important to each of us, but for different reasons that speak to the very important but diverse stakes in psychedelic research today. As such, trial design has operated as a boundary object organising our conversations. Over the year, we discussed readings on the topics of placebo, colonialism, regulatory guidelines, psychotherapy, shamanism and real-world evidence. So far, we have sought to better comprehend one another’s approaches, problems, and modes of inquiry. In what follows we will share some of the questions that have emerged in our group over the year.
Blinding, trial design, and doing “good science” amidst the hype
2022 featured much discussion about the hype bubble that had created sector-wide investment volatility and premature enthusiasm about clinical trial outcomes, alongside concern about methodological rigour. The number of registered clinical trials has exploded, raising concerns about poorly-designed, underpowered trials, the use of questionable research practices, and a lack of transparency. These concerns exist despite a generally enthusiastic embrace of the rhetoric of ‘open science’ (see Jesse, 2017; Petranker et al., 2020). Low quality trial data may well come to pervade the clinical trial space. The resulting confusion will also do work of its own, as poorly powered and inadequately controlled studies of highly selected groups show outlandishly large effect sizes. Our group approached this through the lens of agnotology – the study of the production of ignorance and doubt – and how confusion and uncertainty act to mobilise resources afresh: “When a situation is uncertain, it demands attention, debate, funding, and most crucially, experts to determine how the situation should be resolved” (McGoey, 2009: 155; see also Hess, 2016).
Foremost amongst the concerns about research rigour has been the pervasive problem of unblinding, even in so-called ‘double blind’ trials (e.g., Aday et al., 2022). Several suggestions have been proposed to improve blinding rates, including intentionally cultivating doubt among the participants, or inducing anaesthesia around the drug administration phase to blind the distinctive perceptual features of the drug. Does it make sense to call trials ‘blinded’ in cases where unblinding is either uncontrolled or unmeasured? Should all unblinded trials be considered contaminated, because they may be driven by expectancy? The jury is still out on the best way to handle blinding issues (see Muthukumaraswamy et al., 2021; Szigeti et al., 2022) and on the question of whether, in the case of psychedelic experiences, it is even coherent to do so. For example, the therapeutic efficacy of the drug may depend on its distinctive and identifiable effects.
Issues of (un)blinding occur at many levels and interplay with different kinds of beliefs. These may include the belief in: the power of a substance as perceived by both user and guide, the efficacy of a particular mechanism of action, the healing encounter, the research setting, the movement to mainstream psychedelics, or the progressive march of science itself. Will the healing efficacy of such beliefs (which tend to get lumped together as ‘placebo effects’) differ across different kinds of intervention (e.g. microdosing vs macrodosing), and for different treatment indications (e.g. psychiatry vs general medicine)? If so, it might be time to retire such overly-blunt binaries as specific vs non-specific, drug vs placebo effects, and blinded vs unblinded trials, in favour of more precise and multi-factorial descriptions of trial design.
Our group also discussed interaction effects between the components of psychedelic therapy interventions. With factorial designs more expensive to power, might we find solutions in adaptive trial designs that weed out trial arms in real time as they start to return poor outcomes? And how can we key into existing cross-disciplinary conversations exploring these issues? As anthropologist Ayo Wahlberg has written of the placebo effect,
“Is it via a conditioned response based on cultural familiarity, a logic of expectation that releases endogenous pharmaceutics (such as the by now infamous endorphins), a hope-generated immunological boost that assists self-healing, or does it indeed remain in the subjective realm as an abreaction-aided restoration of cognitive order which enhances the well-being of a patient?” (2008: 86)
Other puzzles abound. For those convinced of the logic of the RCT, how standardised does psychedelic-assisted therapy (PAT) need to be for the trial results to be meaningful? What are the key elements of the therapy? How high are the ‘researcher allegiance biases’ of current trials – particularly given that unblinding may often be shared between participants, therapists and researchers?
Critiquing magic bullets
Systematically manipulating the (ostensibly) non-drug components to see what works and for whom opens up the possibility of ‘culture-controlled trials’ (Wallace, 1959; Langlitz, 2012), a counter to the usual focus on placebo-controlled trials. Such a reframing has been celebrated for having the potential to liberate us from pharmacologicalism (deGrandpre, 2006), a view of drugs as having a pharmacological essence that determined human experience and behaviour upon drug ingestion. It refocuses attention on context, the container, and care as central to modes of healing. One danger lies in building trials that are insufficiently powered to show that such extra-pharmacological factors as the level of training of the therapists, or the degree or kind of preparatory or integration procedures, matter. Poorly-conducted and underpowered decomposition studies may justify paring down interventions further, or even suggest we can cast these variables to the side, as superfluous vestiges of PAT tradition. Advocates of culture-controlled trials could guard against this by resourcing large enough trials to be sensitive to any clinically meaningful effect that non-pharmacological factors produce. But who will fund such trials?
Alternatively, considering how often the efficacy claims born of RCTs fall apart once interventions enter the ‘real world’ (e.g., Cartwright, 2011), why not then just skip this step? Might it be possible to replace clinical trials with natural experiments scaffolded by real-world data gathering? Such uncontrolled studies are the hallmark of methods ranging from case studies, through ethnography, to machine learning using large data sets. Could such approaches enable multi-factorial predictive models that avoid the ‘magic bullet’ narrative, even if at the cost of causal explanation in terms of discrete mechanisms? Indeed, there has been some appetite in our group to pen a call for regulators to overhaul the drug approval process so that it centres real-world evidence approaches. Can we embrace the kind of deep learning from specificity that ethnographic and case study evidence offer, rather than criticising their findings as too location-based? For large data set analysis, how should we deal with proprietary data capture (see Zuboff, 2019) and the black-boxing of variables and associations that are objectionable on ethical grounds (see Benjamin, 2019)?
This discussion raises the question, does it matter how much of the effect can be attributed to the different components of the intervention anyway? Contrary to the habitual practices of researchers and regulators, some therapists and advocates of legalisation celebrate leaving behind such questions as: ‘how much of the effect size is drug vs placebo vs context?’ This line of reasoning requires accepting that we are condemned to see the efficacy of PAT shift with the times – including as the hype around the interventions waxes and wanes, and dominant narratives shape understandings and expectations. Are we content with such a situation? And how might this in turn affect the safety profile of PAT, given safety and efficacy are intertwined in complex ways?
Safety concerns are not being sufficiently studied and reported as it is. The accelerating number of trials and increasing number of avenues for accessing PAT are making safety questions all the more pressing. Financial and reputational stakes encourage the underreporting of adverse events, bringing to mind pressures common across pharmaceutical research, for instance in the hiding of suicidality in SSRI research behind codes such as ‘emotional lability’ (Healy, 2006). What can be learned from this long history? We could build structured tools for reporting adverse events across studies, and pipelines for participant as well as researcher reporting. Harm reduction initiatives could be more systematically resourced and studied, particularly in the growing number of jurisdictions serving as natural experiments for the collection of real-world data. Although secondary analyses and in-depth qualitative reports of particular cases from amongst the clinical trials participants are valuable and needed, these ought to be tempered with consideration of sustained anonymity and ongoing consent in this over-hyped field with its small sample sizes.1
With growing attention to the role of psychotherapy in PAT, 2022 has seen increased calls for psychedelic clinical researchers to learn from the attempts to evidence psychotherapy’s safety and efficacy, including the pitfalls of doing such research. As a group, we have been keen to build more bridges to expertise from the field of psychotherapy, whether to be inspired by the trials and tribulations of generating high quality evidence of psychotherapy’s safety and efficacy, or to help round out – and even update – PAT’s own, perhaps ossified, models of the mind.
Critiquing pharmaceutical capitalism
For those seeking approval for PAT interventions, rather than understanding component mechanisms, could we focus on testing the very therapeutic packages we would like to see scaled? Can, for instance, peer support be built into interventions from the get-go – not with an aim to strip them out in late-stage trials but in order to also scale up this extra-pharmacological factor? And can the risks of such scaling – for example, the professionalisation of the peer role (Faulkner and Kalathil, 2011) – be anticipated and elaborated through fruitful dialogue with existing expertise – here, the critiques of peer support coming out of grassroots mental health consumer/service user and psychiatric survivor movements?
What role could regulators play in the maturing of the research field? In our group we have wondered to what extent regulators are hamstrung by their standard operating practices, from their insistence on placebo-controlled blinded RCT data for drug approval, to their choices of preferred symptom measurement scales (cf. Fried et al., 2022). We have found the difference between investigator-initiated and drug development registration trials of increasing interest – the former seem to be afforded greater methodological latitude, but the evidence they produce does not directly lead to legislative or policy change around drugs. Where does this distinction come from, what sustains it, and what does the distinction in turn enable?
And where – in all of the above – could there be deeper learning from underground and Indigenous traditions of psychedelic use? Our group has barely scratched the surface of what these overlapping traditions might teach those of us designing clinical trials – for instance, about the non-individual dimensions of psychedelic therapy models, or unpacking the assumptions undergirding the dominant clinical research protocols (such as the widespread maxim to ‘trust, be open, let go’). Is it possible to reimagine clinical research on the ground of non-Western foundations in ways that are neither reductive nor appropriative? And what might PAT research learn about process from underground and Indigenous traditions’ own processes of inquiry?
We’re looking forward to more discussions next year!
Aday JS, Heifets BD, Pratscher SD, Bradley E, Rosen R, Woolley JD. (2022) ‘Great Expectations: recommendations for improving the methodological rigor of psychedelic clinical trials’, Psychopharmacology, vol. 239(6):1989-2010. doi: 10.1007/s00213-022-06123-7.
Benjamin R. (2019) Race After Technology: Abolitionist Tools for the New Jim Code, Polity Press.
Cartwright, N. (2011) ‘The Art of Medicine: A philosopher’s view of the long road from RCTs to effectiveness’, The Lancet, vol. 377(9775): 1400-1401.
De Grandpre R. (2006) The Cult of Pharmacology: How America became the world’s most troubled drug culture, Duke University Press.
Dumit J, Sanabria E. (2022) ‘Set, Setting, and Clinical Trials: Colonial Technologies and Psychedelics’, in The Palgrave Handbook of the Anthropology of Technology (eds: MH Bruun, A Wahlberg, R Douglas-Jones, C Hass, K Hoeyer, DB Kristensen, and BR Winthereik), Singapore: Springer, pp.291-308.
Eisner B. (1997) ‘Set, Setting, and Matrix’, Journal of Psychoactive Drugs, 29(2): 213-216, DOI: 10.1080/02791072.1997.10400190.
Faulkner A, Kalathil J. (2012) ‘The Freedom to Be, the Chance to Dream: Preserving user-led peer support in mental health’, London: Together.
Fried EI, Flake JK & Robinaugh DJ. (2022) ‘Revisiting the theoretical and methodological foundations of depression measurement’, Nat Rev Psychol vol. 1, pp.358–368. https://doi.org/10.1038/s44159-022-00050-2
Healy, D. (2006) Let Them Eat Prozac: The unhealthy relationship between the pharmaceutical industry and depression, NYU Press.
Hess, DJ (2016) Undone Science: Social Movements, Mobilized Publics, and Industrial Transitions, The MIT Press.
Jesse B. (2017) ‘Statement on Open Science and Open Praxis with Psilocybin, MDMA, and Similar Substances’, available at https://files.csp.org/open.pdf.
Langlitz N. (2013) Neuropsychedelia: The Revival of Hallucinogen Research Since the Decade of the Brain, University of California Press.
McGoey L. (2009) ‘Pharmaceutical Controversies and the Performative Value of Uncertainty’, Science as Culture, vol. 18(2), pp.151-164.
Muthukumaraswamy S, Forsyth A. and Lumley T. (2021) ‘Blinding and expectancy confounds in psychedelic randomised controlled trials’, Expert Rev Clin Pharmacol, vol.14(9), pp.1133-1152.
Petranker R, Anderson T, Farb N. (2020) ‘Psychedelic Research and the Need for Transparency: Polishing Alice’s Looking Glass’, Front Psychol, vol.11, pp.1681. doi: 10.3389/fpsyg.2020.01681. PMID: 32754101; PMCID: PMC7367180.
Sharma T, Guski LS, Freund N and Gøtzsche PC. (2016) ‘Suicidality and aggression during antidepressant treatment: Systematic review and meta-analyses based on clinical study reports’, BMJ, vol. 352, i65.
Szigeti B, Nutt D, Carhart-Harris R, & Erritzoe D. (2022) ‘On the fallibility of the placebo control and how to address it: a case study in psychedelic microdosing’, https://doi.org/10.31234/osf.io/cjfb6.
Walhberg, A. (2008) ‘Above and beyond superstition — western herbal medicine and the decriminalizing of placebo’, History of the Human Sciences, 21(1), pp.77-101. DOI: 10.1177/0952695107086153.
Wallace AFC. (1959) ‘Cultural Determinants of Response to Hallucinatory Experience’, Archives of General Psychiatry, vol. 1(1), pp. 58–69.
Zuboff, S. (2019) The Age of Surveillance Capitalism: The Fight for a Human Future at the New Frontier of Power. Profile Books.
Cite This Article
Noorani, T., Baggott, M., Bedi, G., Heifets, B., Hendy, K., Langlitz, N., Morgan, C., Sanabria, E., & de Wit, H. (2023, February 2). A Perspective from the ‘Reimagining Psychedelic Trials’ Working Group – Psychedelic Alpha. Psychedelic Alpha. https://psychedelicalpha.com/news/a-perspective-from-the-reimagining-psychedelic-trials-working-group
Part of our Year in Review series
This content is part of our 2022 Year in Review, which looks back at the past year through commentary and analysis, interviews and guest contributions.
Receive New Sections in Your Inbox
To receive future sections of the Review in your inbox, join our newsletter…