Truly Open Science Needs Knowledge Synthesis
This article was written as part of the Future of Open Science Policy project, a partnership between the Federation of American Scientists, the Center for Open Science, and the Wilson Center. This project aims to crowdsource innovative policy proposals that chart a course for the next decade of federal open science. To read the other articles in the series, and to submit a policy idea of your own, please visit the project page.
Ten years on from the Office of Science and Technology Policy’s 2013 public access memo, federally funded scientific papers and data are more available than ever before. Yet as we look towards the future of open science — and open science policy — it is crucial to recognize that truly open science requires that scientists, stakeholders, and the public are not only able to access the products of research, but the knowledge and insights embedded within those products. Given the ever-increasing quantity and complexity of scientific output, this calls for a new focus on synthesis and communication.
Beyond Open Access
Providing the public with access to cutting edge scientific research is a vital goal of open science, and U.S. policy, and has empowered people around the world to better understand the issues that are most important to their health and flourishing. Yet in many cases, the availability of scientific papers themselves is insufficient, or even counterproductive, for ensuring understanding and usability of state-of-the-art knowledge.
To take one example, the possibility that psychedelics will prove to be effective treatments for mental health disorders has garnered perhaps the most public attention of any psychiatric research area in recent decades. Individual papers have attracted extensive media coverage, and their availability to practitioners and the public is critical. But because of the field’s rapidly growing knowledge base and the unclear implications of individual studies, many scientists have called for the public to withhold judgment until more is known. Given this topic’s importance to public and medical stakeholders, and the potential for pervasive coverage to lead to unregulated self-treatment, there is a clear need for expert-driven, clearly communicated, and up-to-the-moment knowledge synthesis.
The idea of advancing the reach and impact of scientific knowledge through aggregation of findings is not new. The ad-hoc production of scientific syntheses by practicing researchers dates back at least a few centuries. In the last few decades, organizations such as the famed Cochrane Collaboration have provided models for standardized, rigorous synthesis within the health and medical sciences, and institutions across various other fields have followed.
A Changing Evidence Landscape
Despite widespread awareness of the value of rigorous, open, and up-to-date evidence synthesis, existing structures are increasingly struggling to keep up with shifting scientific processes. Classic approaches to discovering and summarizing research findings on a given topic (i.e., systematic review and meta-analysis) often take over a year to produce and rapidly go out of date once published. When a field is fast-moving, a lack of up-to-date evidence aggregation leads to less efficient science and hinders evidence-based decision making. Additionally, the nature of scientific outputs themselves are rapidly changing — with innovative approaches for publication, improved standards for credibility, and changing academic incentive structures. These changes require a nimble synthesis regime.
New models for evidence aggregation and communication show promise in strengthening the ecosystem. The TRUST Initiative, for example, demonstrated the potential to embed measures of transparency and credibility into policy-relevant research synthesis, and the Living Evidence model provides a new framework for shifting synthesis away from a static – and often redundant – exercise, to a collaborative and ongoing process embedded within diverse partnerships.
The Need for Government Efforts
These developments signal a clear need for robust resources and capacity for evidence synthesis, yet the ecosystem faces barriers to its sustainability. Indeed, Cochrane, arguably the world leader in trusted medical reviews, recently lost roughly $5 million in funding from the UK’s National Institute for Health and Care Research, and a forthcoming shift towards open access reviews has complicated their financial picture. In the US, a collection of federal evidence clearinghouses must work hard to secure and maintain sufficient political support and resources for their vital work. In general, fast-moving technologies, slow-moving statutory constraints, and a precarious funding landscape mean that important knowledge remains too often scattered across individual studies and outdated reviews.
Much work can and should be done within the academy, industry, and non-governmental institutions. Yet federal actors hold great power – and great responsibility – to advance the cause of trustworthy and up-to-date synthesis and communication of scientific knowledge. Existing efforts show great promise, and span extramural funding (e.g., the NSF’s Opportunities for Promoting Understanding through Synthesis [OPUS] program and the inter-agency Prototype Open Knowledge Network program), organizing/contracting expert-led syntheses (e.g., Office of Disease Preventions’ U.S. Preventive Services Task Force (USPSTF) and the cross-agency evidence clearinghouses), and efforts to generate, synthesize, and apply evidence within government (e.g., agency learning agendas and evaluation plans)
Call to Action
The Year of Open Science provides an important window to both strengthen existing efforts to promote open knowledge and launch ambitious new ones. To meet this moment, we need a broader set of voices contributing ideas on this aspect of open science and countless others. That is why we recently launched an Open Science Policy Sprint, in partnership with the Center for Open Science and the Wilson Center. If you have ideas for federal actions that can help the US meet and exceed its open science goals, we encourage you to submit your proposals here.
To bring participatory science into the mainstream, there will need to be creative policy solutions for incentive mechanisms, standards, funding streams, training ecosystems, assessment mechanisms, and organizational capacity.
Government-sponsored scientific research published in expensive journals should become more readily accessible to the public under an initiative announced by the White House Office of Science and Technology Policy on Friday. Federal agencies that fund at least $100 million per year in scientific research were directed by White House science advisor John Holdren to develop […]