FAS Launches ‘FRO-casting’ Tournament on Metaculus to Solicit Ideas for Focused Research Organizations (FROs)
FROs focus efforts on technology challenges that require coordinated and cross-disciplinary pursuits at the boundary of research and engineering
The Federation of American Scientists (FAS), one of the oldest science think tanks in Washington, and Metaculus, a forecasting and modeling platform, today announced the opening of the ‘FRO-casting’ tournament to test a new approach for evaluating scientific proposals.
This pilot program applies an expected value the forecasting methodology developed by FAS and Metaculus to proposals for Focused Research Organizations (FROs) in the life sciences, sourced by Convergent Research. FROs are envisioned as standalone, time-limited non-profit organizations organized like a startup to solve well-defined technical challenges that are neither profitable nor publishable. Their goal is to produce vital public goods: processes, tools, and datasets that are actively translated into use by others, to enable new methods and accelerate the pace of scientific research.
This is a public tournament in which forecasts will be produced by the Metaculus community and subject-matter experts identified by FAS.
The ‘FRO-casting’ challenge is open to subject matter experts, scientists, forecasters, decision makers, and the public. It is free to participate.
This effort aims to provide (a) quantitative assessments of the risk-reward profile of each FRO proposal that can inform agency decision-making, (b) actionable insight to proposal authors regarding their approach, and (c) new metascientific understanding of forecasting in scientific review.
“It is difficult to balance science agencies’ dual mission of protecting government funding from being spent on overly risky investments while also being ambitious in funding proposals that will push the frontiers of science,” writes FAS Senior Associate Alice Wu, “We at FAS are exploring innovative approaches to peer review to help tackle this challenge.” Ms. Wu is one of the leaders of this challenge, along with FAS colleague Jordan Dworkin, Metascience Program Lead.
By better harnessing the power of data, we can build a learning healthcare system where outcomes drive continuous improvement and where healthcare value leads the way.
In this unprecedented inflection point (and time of difficult disruption) for higher education, science funding, and agency structure, we have an opportunity to move beyond incremental changes and advocate for bold, new ideas that envision a future of the scientific research enterprise that looks very different from the current system.
Assigning persistent digital identifiers (Digital Object Identifiers, or DOIs) and using ORCIDs (Open Researcher and Contributor IDs) for key personnel to track outputs for research grants will improve the accountability and transparency of federal investments in research and reduce reporting burden.
Research funding agencies should apply the content of grant applications to AI tools to predict the future of scientific and technological breakthroughs, enhance peer review, and encourage better research investment decisions by both the public and the private sector.