The National Science Foundation (NSF) has long supported innovative scientific research through grant programs. Among these, the EAGER (Early-concept Grants for Exploratory Research) and RAPID (Rapid Response Research) grants are crucial in fostering early-stage questions and ideas. This memo proposes expanding and improving these programs by addressing their current limitations and leveraging the successful aspects of their predecessor program, the Small Grants for Exploratory Research (SGER) program, and other innovative funding models like the Defense Advanced Research Projects Agency (DARPA).

Current Challenges and Opportunities

The landscape of scientific funding has always been a balancing act between supporting established research and nurturing new ideas. Over the years, the NSF has played a pivotal role in maintaining this balance through various grant programs. One way they support new ideas is through small, fast grants. The SGER program, active from 1990 to 2006, provided nearly 5,000 grants, with an average size of about $54,000. This program laid the groundwork for the current EAGER and RAPID grants, which took SGER’s place and were designed to support exploratory and urgent research, respectively. Using the historical data, researchers analyzed the effectiveness of the SGER program and found it wildly effective, with “transformative research results tied to more than 10% of projects.” The paper also found that the program was underutilized by NSF program officers, leaving open questions about how such an effective and relatively inexpensive mechanism was being overlooked.

Did the NSF learn anything from the paper? Probably not enough, according to the data.

In 2013, the year the SGER paper was published, roughly 2% of total NSF grant funding went towards EAGER and RAPID grants (which translated to more than 4% of the total NSF-funded projects that year). Except for a spike in RAPID grants in 2020 in response to the COVID-19 pandemic, there has been a steady decline in the volume, amount, and percentage of EAGER and RAPID grants over the ensuing decade. Over the past few years, EAGER and RAPID have barely exceeded 1% of the award budget. Despite the proven effectiveness of these funding mechanisms and their relative affordability, the rate of small, fast grantmaking has stagnated over the past decade.

There is a pressing need to support more high-risk, high-reward research through more flexible and efficient funding mechanisms. Increasing the small, fast grant capacity of the national research programs is an obvious place to improve, given the results of the SGER study and the fact that small grants are easier on the budget.

The current EAGER and RAPID grant programs, while effective, face administrative and cultural challenges that limit their scalability and impact. The reasons for their underuse remain poorly understood, but anecdotal insights from NSF program officers offer clues. The most plausible explanation is also the simplest: It’s difficult to prioritize small grants while juggling larger ones that carry higher stakes and greater visibility. While deeper, formal studies could further pinpoint the barriers, the lack of such research should not hinder the pursuit of bold, alternative strategies—especially when small grant programs offer a rare blend of impact and affordability.

Drawing inspiration from the ARPA model, which empowers program managers with funding discretion and contracting authority, there is an opportunity to revolutionize how small grants are administered. The ARPA approach, characterized by high degrees of autonomy and focus on high-risk, high-reward projects, has already inspired successful initiatives beyond its initial form in the Department of Defense (DARPA), like ARPA-E for energy and ARPA-H for health. A similar “Micro-ARPA” approach — in which dedicated, empowered personnel manage these funds — could be transformative for ensuring that small grant programs within NSF reach their full potential. 

Plan of Action

To enhance the volume, impact, and efficiency of small, fast grant programs, we propose the following:

1. Establish a Micro-ARPA program with dedicated funding for small, flexible grants: The NSF should allocate 50% of the typical yearly funding for EAGER/RAPID grants — roughly $50–100 million per year — to a separate dedicated fund. This fund would use the existing EAGER/RAPID mechanisms for disbursing awards but be implemented through a programmatically distinct Micro-ARPA model that empowers dedicated project managers with more discretion and reduces the inherent tension between use of these streamlined mechanisms and traditional applications.
   1. By allocating approximately 50% of the current spend to this fund and using the existing EAGER/RAPID mechanisms within it, this fund would be unlikely to pull resources from other programs. It would instead set a floor for the use of these flexible frameworks while continuing to allow for their use in the traditional program-level manner when desired.
2. Establish a Micro-ARPA program manager (PM) role: As compared to the current model, in which the allocation of EAGER/RAPID grants is a small subset of broader NSF program director responsibilities, Micro-ARPA PMs (who could be lovingly nicknamed “Micro-Managers”) should be hired or assigned within each directorate to manage the dedicated Micro-ARPA budgets. Allocating these small, fast grants should be their only job in the directorate, though it can and should be a part-time position per the needs of the directorate.
   1. Given the diversity of awards and domains that this officer may consider, they should be empowered to seek the advice of program-specific staff within their directorate as well as external reviewers when they see fit, but should not be required to make funding decisions in alignment with programmatic feedback. 
   2. Applications to the Micro-ARPA PM role should be competitive and open to scientists and researchers at all career levels. Based on our experience managing these programs at the Experiment Foundation, there is every reason to suspect that early-career researchers, community-based researchers, or other innovators from nontraditional backgrounds could be as good or better than experienced program officers. Given the relatively low cost of the program, the NSF should open this role to a wide variety of participants to learn and study the outcomes.
3. Evaluate: The agency should work with academic partners to design and implement clear metrics—similar to those used in the paper that evaluated the SGER program—to assess the programs’ decision-making and impacts. Findings should be regularly compiled and circulated to PMs to facilitate rapid learning and improvement. Based on evaluation of this program, and comparison to the existing approach to allocating EAGER/RAPID grants, relative funding quantities between the two can be reallocated to maximize scientific and social impact. 

Benefits

The proposed enhancements to the small grant programs will yield several key benefits:

1. Increased innovation: By funding more early-stage, high-risk projects, we can accelerate scientific breakthroughs and technological advancements, addressing global challenges more effectively.
2. Support for early-career scientists: Expanded grant opportunities will empower more early-career researchers to pursue innovative ideas, fostering a new generation of scientific leaders.
3. Experience opportunity for program managers: Running Micro-ARPAs will provide an opportunity for new and emerging program manager talent to train and develop their skills with relatively smaller amounts of money.
4. Platform for metascience research: The high volume of new Micro-ARPA PMs will create an opportunity to study the effective characteristics of program managers and translate them into insights for larger ARPA programs.
5. Administrative efficiency: A streamlined, decentralized approach will reduce the administrative burden on both applicants and program officers, making the grant process more agile and responsive. Speedier grants could also help the NSF achieve its stated dwell time goal of 75% (response rate within six months), which they have failed to do consistently in recent years.

Conclusion

Small, fast grant programs are vital to supporting transformative research. By adopting a more flexible, decentralized model, we can significantly enhance their impact. The proposed changes will foster a more dynamic and innovative scientific ecosystem, ultimately driving progress and addressing urgent global challenges.

This action-ready policy memo is part of Day One 2025 — our effort to bring forward bold policy ideas, grounded in science and evidence, that can tackle the country’s biggest challenges and bring us closer to the prosperous, equitable and safe future that we all hope for whoever takes office in 2025 and beyond.

Frequently Asked Questions

Do small grants really matter?

Absolutely. The research supports it, but the stories bring it to life. Ask any scientist about the first grant they received for their own work, and you’ll often hear about a small, pivotal award that changed everything. These grants may not make headlines, but they ignite careers, foster innovation, and open doors to discovery.

read more

Can this be done with reallocating existing budget and under existing authority?

Almost certainly within the existing budget. As for authority, it’s theoretically possible but politically fraught. NSF program officers already have the discretion to use RAPID and EAGER grants as they see fit, so in principle, a program officer could be directed to use only those mechanisms. That mandate would essentially transform their role into a Micro-ARPA program manager. The real challenge lies in the culture and practice of grant-making. There’s a reason that DARPA operates independently from the rest of the military branches’ research and development infrastructure.

read more

Why would dedicated staffing and a Micro-ARPA program structure overcome administrative challenges?

In a word: focus. Program officers juggle large, complex grants that demand significant time and resources. Small grants, though impactful, can get lost in the shuffle. By dedicating staff to exclusively manage these smaller, fast grants, we create the conditions to test an important hypothesis: that administrative burden and competing priorities, not lack of interest, are the primary barriers to scaling small grant programs. It’s about clearing the runway so these grants can truly take off.

read more

Why not just set goals for greater usage of EAGER and RAPID?

Encouraging greater use of EAGER and RAPID is a good start, but it’s not enough. We need to think bigger, trying alternative structures and dedicated programs that push the boundaries of what’s possible. Incremental change can help, but bold experiments are what transform systems.

read more

Artificial intelligence (AI) has transformative potential in the public health space – in an era when millions of Americans have limited access to high-quality healthcare services, AI-based tools and applications can enable remote diagnostics, drive efficiencies in implementation of public health interventions, and support clinical decision-making in low-resource settings. However, innovation driven primarily by the private sector today may be exacerbating existing disparities by training models on homogenous datasets and building tools that primarily benefit high socioeconomic status (SES) populations. 

To address this gap, the Center for Medicare and Medicaid Innovation (CMMI) should create an AI for Medicaid (AIM) Initiative to distribute competitive grants to state Medicaid programs (in partnership with the private sector) for pilot AI solutions that lower costs and improve care delivery for rural and low-income populations covered by Medicaid. 

Challenge & Opportunity

In 2022, the United States spent $4.5 trillion on healthcare, accounting for 17.3% of total GDP. Despite spending far more on healthcare per capita compared to other high-income countries, the United States has significantly worse outcomes, including lower life expectancy, higher death rates due to avoidable causes, and lesser access to healthcare services. Further, the 80 million low-income Americans reliant on state-administered Medicaid programs often have below-average health outcomes and the least access to healthcare services. 

AI has the potential to transform the healthcare system – but innovation solely driven by the private sector results in the exacerbation of the previously described inequities. Algorithms in general are often trained on datasets that do not represent the underlying population – in many cases, these training biases result in tools and models that perform poorly for racial minorities, people living with comorbidities, and people of low SES. For example, until January 2023, the model used to prioritize patients for kidney transplants systematically ranked Black patients lower than White patients – the race component was identified and removed due to advocacy efforts within the medical community. AI models, while significantly more powerful than traditional predictive algorithms, are also more difficult to understand and engineer, resulting in the likelihood of further perpetuating such biases. 

Additionally, startups innovating the digital health space today are not incentivized to develop solutions for marginalized populations. For example, in FY 2022, the top 10 startups focused on Medicaid received only $1.5B in private funding, while their Medicare Advantage (MA)-focused counterparts received over $20B. Medicaid’s lower margins are not attractive to investors, so digital health development targets populations that are already well-insured and have higher degrees of access to care.

The Federal Government is uniquely positioned to bridge the incentive gap between developers of AI-based tools in the private sector and American communities who would benefit most from said tools. Accordingly, the Center for Medicare and Medicaid Innovation (CMMI) should launch the AI for Medicaid (AIM) Initiative to incentivize and pilot novel AI healthcare tools and solutions targeting Medicaid recipients. Precedents in other countries demonstrate early success in state incentives unlocking health AI innovations – in 2023, the United Kingdom’s National Health Service (NHS) partnered with Deep Medical to pilot AI software that streamlines services by predicting and mitigating missed appointment risk. The successful pilot is now being adopted more broadly and is projected to save the NHS over $30M annually in the coming years. 

The AIM Initiative, guided by the structure of the former Medicaid Innovation Accelerator Program (IAP), President Biden’s executive order on integrating equity into AI development, and HHS’ Equity Plan (2022), will encourage the private sector to partner with State Medicaid programs on solutions that benefit rural and low-income Americans covered by Medicaid and drive efficiencies in the overall healthcare system. 

Plan of Action

CMMI will launch and operate the AIM Initiative within the Department of Health and Human Services (HHS). $20M of HHS’ annual budget request will be allocated towards the program. State Medicaid programs, in partnership with the private sector, will be invited to submit proposals for competitive grants. In addition to funding, CMMI will leverage the former structure of the Medicaid IAP program to provide state Medicaid agencies with technical assistance throughout their participation in the AIM Initiative. The programs ultimately selected for pilot funding will be monitored and evaluated for broader implementation in the future. 

Sample Detailed Timeline

• 0-6 months:
  • HHS Secretary to announce and launch the AI for Medicaid (AIM) Initiative within CMMI (e.g., delineating personnel responsibilities and engaging with stakeholders to shape the program)
  • HHS to include AIM funding in annual budget request to Congress ($20M allocation)

• 6-12 months:
  • CMMI to engage directly with state Medicaid agencies to support proposal development and facilitate connections with private sector partners
  • CMMI to complete solicitation period and select ~7-10 proposals for pilot funding of ~$2-5M each by end of Year 1
• Year 2-7: Launch and roll out selected AI projects, led by state Medicaid agencies with continued technical assistance from CMMI
• Year 8: CMMI to produce an evaluative report and provide recommendations for broader adoption of AI tools and solutions within Medicaid-covered and other populations

Risks and Limitations

• Participation: Success of the initiative relies on state Medicaid programs and private sector partners’ participation. To mitigate this risk, CMMI will engage early with the National Association of Medicaid Directors (NAMD) to generate interest and provide technical assistance in proposal development. These conversations will also include input and support from the HHS Office of the Chief AI Officer (OCAIO) and its AI Council/Community of Practice. Further, startups in the healthcare AI space will be invited to engage with CMMI on identifying potential partnerships with state Medicaid agencies. A secondary goal of the initiative will be to ensure a number of private sector partners are involved in AIM.
• Oversight: AI is at the frontier of technological development today, and it is critical to ensure guardrails are in place to protect patients using AI technologies from potential adverse outcomes. To mitigate this risk, state Medicaid agencies will be required to submit detailed evaluation plans with their proposals. Additionally, informed consent and the ability to opt-out of data sharing when engaging with personally identifiable information (PII) and diagnostic or therapeutic technologies will be required. Technology partners (whether private, academic, or public sector) will further be required to demonstrate (1) adequate testing to identify and reduce bias in their AI tools to reasonable standards, (2) engagement with beneficiaries in the development process, and (3) leveraging testing environments that reflect the particular context of the Medicaid population. Finally, all proposals must adhere to guidelines published by AI guidelines adopted by HHS and the federal government more broadly, such as the CMS AI Playbook, the HHS Trustworthy AI Playbook, and any imminent regulations.
• Longevity: As a pilot grant program, the initiative does not promise long-term results for the broader population and will only facilitate short-term projects at the state level. Consequently, HHS leadership must remain committed to program evaluation and a long-term outlook on how AI can be integrated to support Americans more broadly. AI technologies or tools considered for acquisition by state Medicaid agencies or federal agencies after pilot implementation should ensure compliance with OMB guidelines.

Conclusion

The AI for Medicaid Initiative is an important step in ensuring the promise of artificial intelligence in healthcare extends to all Americans. The initiative will enable the piloting of a range of solutions at a relatively low cost, engage with stakeholders across the public and private sectors, and position the United States as a leader in healthcare AI technologies. Leveraging state incentives to address a critical market failure in the digital health space can additionally unlock significant efficiencies within the Medicaid program and the broader healthcare system. The rural and low-income Americans reliant on Medicaid have too often been an afterthought in access to healthcare services and technologies – the AIM Initiative provides an opportunity to address this health equity gap.

This action-ready policy memo is part of Day One 2025 — our effort to bring forward bold policy ideas, grounded in science and evidence, that can tackle the country’s biggest challenges and bring us closer to the prosperous, equitable and safe future that we all hope for whoever takes office in 2025 and beyond.

In 2020, we outlined a vision for how the incoming presidential administration could strengthen the nation’s innovation ecosystem, encouraging the development and commercialization of science and technology (S&T) based ventures. This vision entailed closing critical gaps from lab to market, with an emphasis on building a broadly inclusive pipeline of entrepreneurial talent while simultaneously providing key support in venture development. 

During the intervening years, we have seen extraordinary progress, in good part due to ambitious legislation. Today, we propose innovative ways that the federal government can successfully build on this progress and make the most of new programs. With targeted policy interventions, we can efficiently and effectively support the U.S. innovation economy through the translation of breakthrough scientific research from the lab to the market. The action steps we propose are predicated on three core principles: inclusion, relevance, and sustainability. Accelerating our innovation economy and expanding access to it can make our nation more globally competitive, increase economic development, address climate change, and improve health outcomes. A strong innovation economy benefits everyone. 

Challenge

Our Day One 2020 memo began by pitching the importance of innovation and entrepreneurship: “Advances in scientific and technological innovations—and, critically, the ability to efficiently transform breakthroughs into scalable businesses—have contributed enormously to American economic leadership over the past century.” Now, it is widely recognized that innovation and entrepreneurship are key to both global economic leadership and addressing the challenges of changing climate. The question is no longer whether we must innovate but rather how effectively we can stimulate and expand a national innovation economy. 

Since 2020, the global and U.S. economies have gone through massive change and uncertainty.  The Global Innovation Index (GII) 2023 described the challenges involved in its yearly analysis of monitoring global innovation trends amid uncertainty brought on by a sluggish economic recovery from the COVID-19 pandemic, elevated interest rates, and geopolitical tensions. Innovation indicators like scientific publications, research and development (R&D), venture capital (VC) investments, and the number of patents rose to historic levels, but the value of VC investment declined by close to 40%. As a counterweight to this extensive uncertainty, the GII 2023 described the future of S&T innovation and progress as “the promise of Digital Age and Deep Science innovation waves and technological progress.” 

In the face of the pressures of global competitiveness, societal needs, and climate change, the clear way forward is to continue to innovate based on scientific and technical advancements. Meeting the challenges of our moment in history requires a comprehensive and multifaceted effort led by the federal government with many public and private partners.

Grow global competitiveness

Around the world, countries are realizing that investing in innovation is the most efficient way to transform their economies. In 2022, the U.S. had the largest R&D budget internationally, with spending growing by 5.6%, but China’s investment in R&D grew by 9.8%. For the U.S. to remain a global economic leader, we must continue to invest in innovation infrastructure, including the basic research and science, technology, engineering, and math (STEM) education that underpins our leadership, while we grow our investments in translational innovation. This includes reframing how existing resources are used as well as allocating new spending. It will require a systems change orientation and long-term commitments. 

Increase economic development

Supporting and growing an innovation economy is one of our best tools for economic development. From place-based innovation programs to investment in emerging research institutions (ERIs) and Minority-Serving Institutions (MSIs) to training S&T innovators to become entrepreneurs in I-Corps™, these initiatives stimulate local economies, create high-quality jobs, and reinvigorate regions of the country left behind for too long. 

Address climate change

In 2023, for the first time, global warming exceeded 1.5°C for an entire year. It is likely that all 12 months of 2024 will also exceed 1.5°C above pre-industrial temperatures. Nationally and internationally, we are experiencing the effects of climate change; climate mitigation, adaptation, and resilience solutions are urgently needed and will bring outsized economic and social impact.

Improve U.S. health outcomes

The COVID-19 pandemic was devastating, particularly impacting underserved and underrepresented populations, but it spurred unprecedented medical innovation and commercialization of new diagnostics, vaccines, and treatments. We must build on this momentum by applying what we’ve learned about rapid innovation to continue to improve U.S. health outcomes and to ensure that our nation’s health care needs across regions and demographics are addressed. 

Make innovation more inclusive

Representational disparities persist across racial/ethnic and gender lines in both access to and participation in innovation and entrepreneurship. This is a massive loss for our innovation economy. The business case for broader inclusion and diversity is growing even stronger, with compelling data tracking the relationship between leadership diversity and company performance. Inclusive innovation is more effective innovation: a multitude of perspectives and lived experiences are required to fully understand complex problems and create truly useful solutions. To reap the full benefits of innovation and entrepreneurship, we must increase access and pathways for all. 

Opportunity

With the new presidential administration in 2025, the federal government has a renewed opportunity to prioritize policies that will generate and activate a wave of powerful, inclusive innovation and entrepreneurship. Implementing such policies and funding the initiatives that result is crucial if we as a nation are to successfully address urgent problems such as the climate crisis and escalating health disparities. 

Our proposed action steps are predicated on three core principles: inclusion, relevance, and sustainability. 

Inclusion

One of this nation’s greatest and most unique strengths is our heterogeneity. We must leverage our diversity to meet the complexity of the substantial social and economic challenges that we face today. The multiplicity of our people, communities, identities, geographies, and lived experiences gives the U.S. an edge in the global innovation economy: When we bring all of these perspectives to the table, we better understand the challenges that we face, and we are better equipped to innovate to meet them. If we are to harness the fullness of our nation’s capacity for imagination, ingenuity, and creative problem-solving, entrepreneurship pathways must be inclusive, equitable, and accessible to all. Moreover, all innovators must learn to embrace complexity, think expansively and critically, and welcome perspectives beyond their own frame of reference. Collaboration and mutually beneficial partnerships are at the heart of inclusive innovation. 

Relevance

Innovators and entrepreneurs have the greatest likelihood of success—and the greatest potential for impact—when their work is purpose-driven, nimble, responsive to consumer needs, and adaptable to different applications and settings.  Research suggests that “breakthrough innovation” occurs when different actors bring complementary and independent skills to co-create interesting solutions to existing problems. Place-based innovation is one strategy to make certain that technology development is grounded in regional concerns and aspirations, leading to better outcomes for all concerned. 

Sustainability 

Multiple layers of sustainability should be integrated into the innovation and entrepreneurship landscape. First and most salient is supporting the development of innovative technologies that respond to the climate crisis and bolster national resilience. Second is encouraging innovators to incorporate sustainable materials and processes in all stages of research and development so that products benefit the planet and risks to the environment are mitigated through the manufacturing process, whether or not climate change is the focus of the technology. Third, it is vital to prioritize helping ventures develop sustainable business models that will result in long-term viability in the marketplace. Fourth, working with innovators to incorporate the potential impact of climate change into their business planning and projections ensures they are equipped to adapt to changing needs. All of these layers contribute to sustaining America’s social well-being and economic prosperity, ensuring that technological breakthroughs are accessible to all.

Proposed Action

Recommendation 1. Supply and prepare talent.

Continuing to grow the nation’s pipeline of S&T innovators and entrepreneurs is essential. Specifically, creating accessible entrepreneurial pathways in STEM will ensure equitable participation. Incentivizing individuals to become innovators-entrepreneurs, especially those from underrepresented groups, will strengthen national competitiveness by leveraging new, untapped potential across innovation ecosystems.

Expand the I-Corps model

By bringing together experienced industry mentors, commercial experts, research talent, and promising technologies, I-Corps teaches scientific innovators how to evaluate whether their innovation can be commercialized and how to take the first practical steps of bringing their product to market. Ten new I-Corps Hubs, launched in 2022, have expanded the network of engaged universities and collaborators, an important step toward growing an inclusive innovation ecosystem across the U.S. 

Interest in I-Corps far outpaces current capacity, and increasing access will create more expansive pathways for underrepresented entrepreneurs. New federal initiatives to support place-based innovation and to grow investment at ERIs and MSIs will be more successful if they also include lab-to-market training programs such as I-Corps. Federal entities should institute policies and programs that increase awareness about and access to sequenced venture support opportunities for S&T innovators. These opportunities should include intentional “de-risking” strategies through training, advising, and mentoring.

Specifically, we recommend expanding I-Corps capacity so that all interested participants can be accommodated. We should also strive to increase access to I-Corps so that programs reach diverse students and researchers. This is essential given the U.S. culture of entrepreneurship that remains insufficiently inclusive of women, people of color, and those from low-income backgrounds, as well as international students and researchers, who often face barriers such as visa issues or a lack of institutional support needed to remain in the U.S. to develop their innovations. Finally, we should expand the scope of what I-Corps offers, so that programs provide follow-on support, funding, and access to mentor and investor networks even beyond the conclusion of initial entrepreneurial training. 

I-Corps has already expanded beyond the National Science Foundation (NSF) to I-Corps at National Institutes of Health (NIH), to empower biomedical entrepreneurs, and Energy I-Corps, established by the Department of Energy (DOE) to accelerate the deployment of energy technologies. We see the opportunity to grow I-Corps further by building on this existing infrastructure and creating cohorts funded by additional science agencies so that more basic research is translated into commercially viable businesses. 

Close opportunity gaps by supporting emerging research institutions (ERIs) and Minority-Serving Institutions (MSIs)

ERIs and MSIs provide pathways to S&T innovation and entrepreneurship, especially for individuals from underrepresented groups. In particular, a VentureWell-commissioned report identified that “MSIs are centers of research that address the unique challenges and opportunities faced by BIPOC communities. The research that takes place at MSIs offers solutions that benefit a broad and diverse audience; it contributes to a deeper understanding of societal issues and drives innovation that addresses these issues.”

The recent codification of ERIs in the 2022 CHIPS and Science Act pulls this category into focus. Defining this group, which comprises thousands of higher education institutions,  was the first step in addressing the inequitable distribution of federal research funding. That imbalance has perpetuated regional disparities and impacted students from underrepresented groups, low-income students, and rural students in particular. Further investment in ERIs will result in more STEM-trained students, who can become innovators and entrepreneurs with training and engagement. Additional support that could be provided to ERIs includes increased research funding, access to capital/investment, capacity building (faculty development, student support services), industry partnerships, access to networks, data collection/benchmarking, and implementing effective translation policies, incentives, and curricula. 

Supporting these institutions—many of which are located in underserved rural or urban communities that experience underinvestment—provides an anchor for sustained talent development and economic growth. 

Recommendation 2. Support place-based innovation.

Place-based innovation not only spurs innovation but also builds resilience in vulnerable communities, enhancing both U.S. economic and national security. Communities that are underserved and underinvested in present vulnerabilities that hostile actors outside of the U.S. can exploit. Place-based innovation builds resilience: innovation creates high-quality jobs and brings energy and hope to communities that have been left behind, leveraging the unique strengths, ecosystems, assets, and needs of specific regions to drive economic growth and address local challenges.  

Evaluate and learn from transformative new investments

There have been historic levels of government investment in place-based innovation, funding the NSF’s Regional Innovation Engines awards and two U.S. Department of Commerce Economic Development Administration (EDA) programs: the Build Back Better Regional Challenge and Regional Technology and Innovation Hubs awards. The next steps are to refine, improve, and evaluate these initiatives as we move forward. 

Unify the evaluation framework, paired with local solutions

Currently, evaluating the effectiveness and outcomes of place-based initiatives is challenging, as benchmarks and metrics can vary by region. We propose a unified framework paired with solutions locally identified by and tailored to the specific needs of the regional innovation ecosystem. A functioning ecosystem cannot be simply overlaid upon a community but must be built by and for that community. The success of these initiatives requires active evaluation and incorporation of these learnings into effective solutions, as well as deep strategic collaboration at the local level, with support and time built into processes.   

Recommendation 3. Increase access to financing and capital.

Funding is the lifeblood of innovation. S&T innovation requires more investment and more time to bring to market than other types of ventures, and early-stage investments in S&T startups are often perceived as risky by those who seek a financial return. Bringing large quantities of early-stage S&T innovations to the point in the commercialization process where substantial private capital takes an interest requires nondilutive and patient government support. The return on investment that the federal government seeks is measured in companies successfully launched, jobs created, and useful technologies brought to market.

Disparities in access to capital by companies owned by women and underrepresented minority founders are well documented. The federal government has an interest in funding innovators and entrepreneurs from many backgrounds: they bring deep and varied knowledge and a multitude of perspectives to their innovations and to their ventures. This results in improved solutions and better products at a cheaper price for consumers. Increasing access to financing and capital is essential to our national economic well-being and to our efforts to build climate resilience. 

Expand SBIR/STTR access and commercial impact

The SBIR and STTR programs spur innovation, bolster U.S. economic competitiveness, and strengthen the small business sector, but barriers persist. In a recent third-party assessment of the SBIR/STTR program at NIH, the second largest administrator of SBIR/STTR funds, the committee found outreach from the SBIR/STTR programs to underserved groups is not coordinated, and there has been little improvement in the share of applications from or awards to these groups in the past 20 years. Further, NIH follows the same processes used for awarding R01 research grants, using the same review criteria and typically the same reviewers, omitting important commercialization considerations. 

To expand access and increase the commercialization potential of the SBIR/STTR program, funding agencies should foster partnerships with a broader group of organizations, conduct targeted outreach to potential applicants, offer additional application assistance to potential applicants, work with partners to develop mentorship and entrepreneur training programs, and increase the percentage of private-sector reviewers with entrepreneurial experience. Successful example programs of SBIR/STTR support programs include the NSF Beat-The-Odds Boot Camp, Michigan’s Emerging Technologies Fund, and the SBIR/STTR Innovation Summit. 

Provide entrepreneurship education and training

Initiatives like NSF Engines, Tech Hubs, Build-Back-Better Regional Challenge, the Minority Business Development Agency (MBDA) Capital Challenge, and the Small Business Administration (SBA) Growth Accelerator Fund expansion will all achieve more substantial results with supplemental training for participants in how to develop and launch a technology-based business. As an example of the potential impact, more than 2,500 teams have participated in I-Corps since the program’s inception in 2012. More than half of these teams, nearly 1,400, have launched startups that have cumulatively raised $3.16 billion in subsequent funding, creating over 11,000 jobs. Now is an opportune moment to widely apply similarly effective approaches. 

Launch a local investment education initiative

Angel investors are typically providing the first private funding available to S&T innovators and entrepreneurs. These very early-stage funders give innovators access to needed capital, networks, and advice to get their ventures off the ground. We recommend that the federal government expand the definition of an accredited investor and incentivize regionally focused initiatives to educate policymakers and other regional stakeholders about best practices to foster more diverse and inclusive angel investment networks. With the right approach and support, there is the potential to engage thousands more high-net-worth individuals in early-stage investing, contributing their expertise and networks as well as their wealth.

Encourage investment in climate solutions

Extreme climate-change-attributed weather events such as floods, hurricanes, drought, wildfire, and heat waves cost the global economy an average of $143 billion annually. S&T innovations have the potential to help address the impacts of climate change at every level:

• Mitigation. Promising new ideas and technologies can slow or even prevent further climate change by reducing or removing greenhouse gasses.
• Adaptation. We can adapt processes and systems to better respond to adverse events, reducing the impacts of climate change. 
• Resilience. By anticipating, preparing for, and responding to hazardous events, trends, or disturbances caused by climate change, we can continue to thrive on our changing planet. 

Given the global scope of the problem and the shared resources of affected communities, the federal government can be a leader in prioritizing, collaborating, and investing in solutions to direct and encourage S&T innovation for climate solutions. There is no question whether climate adaptation technologies will be needed, but we must ensure that these solutions are technologies that create economic opportunity in the U.S. We encourage the expansion and regular appropriations of funding for successful climate programs across federal agencies, including the DoE Office of Technology Transitions’ Energy Program for Innovation Clusters, the National Oceanic and Atmospheric Administration’s (NOAA) Ocean-Based Climate Resilience Accelerators program, and the U.S. Department of Agriculture’s Climate Hubs. 

Recommendation 4. Shift to a systems change orientation.

To truly stimulate a national innovation economy, we need long-term commitments in policy, practice, and regulations. Leadership and coordination from the executive branch of the federal government are essential to continue the positive actions already begun by the Biden-Harris Administration.  

These initiatives include: 

• Scientific integrity and evidence-based policy-making memo
• Catalyzing Clean Energy Industry Executive Order
• Implementation of the Infrastructure Investment and Jobs Act
• Advancing Biotechnology and Biomanufacturing Innovation for a Sustainable, Safe, and Secure American Bioeconomy
• Implementation of the CHIPS Act of 2022
• Advancing Women’s Health Research and Innovation

Policy

Signature initiatives like the CHIPS and Science Act, Infrastructure Investment and Jobs Act, and the National Quantum Initiative Act are already threatened by looming appropriations shortfalls. We need to fully fund existing legislation, with a focus on innovative and translational R&D. According to a report by PricewaterhouseCoopers, if the U.S. increased federal R&D spending to 1% of GDP by 2030, the nation could support 3.4 million jobs and add $301 billion in labor income, $478 billion in economic value, and $81 billion in tax revenue. Beyond funding, we propose supporting innovative policies to bolster U.S. innovation capacity at the local and national levels. This includes providing R&D tax credits to spur research collaboration between industry and universities and labs, providing federal matching funds for state and regional technology transfer and commercialization efforts, and revising the tax code to support innovation by research-intensive, pre-revenue companies.

Practice

The University and Small Business Patent Procedures Act of 1980, commonly known as the Bayh-Dole Act, allows recipients of federal research funding to retain rights to inventions conceived or developed with that funding. The academic tech transfer system created by the Bayh-Dole Act (codified as amended at 35 U.S.C. §§ 200-212) generated nearly $1.3 trillion in economic output, supported over 4.2 million jobs, and launched over 11,000 startups. We should preserve the Bayh-Dole Act as a means to promote commercialization and prohibit the consideration of specific factors, such as price, in march-in determinations. 

In addition to the continual practice and implementation of successful laws such as Bayh-Dole, we must repurpose resources to support innovation and the high-value jobs that result from S&T innovation. We believe the new administration should allocate a share of federal funding to promote technology transfer and commercialization and better incentivize commercialization activities at federal labs and research institutes. This could include new programs such as mentoring programs for researcher entrepreneurs and student entrepreneurship training programs. Incentives include evaluating the economic impact of lab-developed technology by measuring commercialization outcomes in the annual Performance Evaluation and Management Plans of federal labs, establishing stronger university entrepreneurship reporting requirements to track and reward universities that create new businesses and startups, and incentivizing universities to focus more on commercialization activities as part of promotion and tenure of faculty, 

Regulations

A common cause of lab-to-market failure is the inability to secure regulatory approval, particularly for novel technologies in nascent industries. Regulation can limit potentially innovative paths, increase innovation costs, and create a compliance burden on businesses that stifle innovation. Regulation can also spur innovation by enabling the management of risk. In 1976 the Cambridge (Massachusetts) City Council became the first jurisdiction to regulate recombinant DNA, issuing the first genetic engineering license and creating the first biotech company. Now Boston/Cambridge is the world’s largest biotech hub: home to over 1,000 biotech companies, 21% of all VC biotech investments, and 15% of the U.S. drug development pipeline.

To advance innovation, we propose two specific regulatory actions:

• Climate. We recommend the Environmental Protection Agency (EPA) adopt market-based strategies to help fight climate change by monitoring and regulating CO₂  emissions, putting an explicit price on carbon emissions, and incentivizing businesses to find cost-effective and innovative ways to reduce those emissions. 
• Health. We recommend strengthening regulatory collaboration between the Food and Drug Administration (FDA) and the Centers for Medicare & Medicaid Services (CMS) to establish a more efficient and timely reimbursement process for novel FDA-authorized medical devices and diagnostics. This includes refining the Medicare Coverage of Innovative Technologies rule and fully implementing the new Transitional Coverage for Emerging Technologies pathway to expedite the review, coverage determination, and reimbursement of novel medical technologies.

Conclusion

To maintain its global leadership role, the United States must invest in the individuals, institutions, and ecosystems critical to a thriving, inclusive innovation economy. This includes mobilizing access, inclusion, and talent through novel entrepreneurship training programs; investing, incentivizing, and building the capacity of our research institutions; and enabling innovation pathways by increasing access to capital, networks, and resources.

Fortunately, there are  several important pieces of legislation recommitting the U.S. leadership to bold S&T goals, although much of the necessary resources are yet to be committed to those efforts. As a society, we benefit when federally supported innovation efforts tackle big problems that are beyond the scope of single ventures; notably, the many challenges arising from climate change. A stronger, more inclusive innovation economy benefits the users of S&T-based innovations, individual innovators, and the nation as a whole.

When we intentionally create pathways to innovation and entrepreneurship for underrepresented individuals, we build on our strengths. In the United States, our strength has always been our people, who bring problem-solving abilities from a multitude of perspectives and settings. We must unleash their entrepreneurial power and become, even more, a country of innovators.. 

Earlier memo contributors Heath Naquin and Shaheen Mamawala (2020) were not involved with this 2024 memo.

This action-ready policy memo is part of Day One 2025 — our effort to bring forward bold policy ideas, grounded in science and evidence, that can tackle the country’s biggest challenges and bring us closer to the prosperous, equitable and safe future that we all hope for whoever takes office in 2025 and beyond.

The federal government transfers approximately $1.1 trillion dollars every year to state and local governments. Yet most states and localities are not evaluating whether the programs deploying these funds are increasing community well-being. Similarly, achieving important national goals like increasing clean energy production and transmission often requires not only congressional but also state and local policy reform. Yet many states and localities are not implementing the evidence-based policy reforms necessary to achieve these goals.

State and local government innovation is a problem not only of politics but also of capacity. State and local governments generally lack the technical capacity to conduct rigorous evaluations of the efficacy of their programs, search for reliable evidence about programs evaluated in other contexts, and implement the evidence-based programs with the highest chances of improving outcomes in their jurisdictions. This lack of capacity severely constrains the ability of state and local governments to use federal funds effectively and to adopt more effective ways of delivering important public goods and services. To date, efforts to increase the use of evaluation evidence in federal agencies (including the passage of the Evidence Act) have not meaningfully supported the production and use of evidence by state and local governments.

Despite an emerging awareness of the importance of state and local government innovation capacity, there is a shortage of plausible strategies to build that capacity. In the words of journalist Ezra Klein, we spend “too much time and energy imagining the policies that a capable government could execute and not nearly enough time imagining how to make a government capable of executing them.”

Yet an emerging body of research is revealing that an effective strategy to build government innovation capacity is to partner government agencies with local universities on scientifically rigorous evaluations of the efficacy of their programs, curated syntheses of reliable evaluation evidence from other contexts, and implementation of evidence-based programs with the best chances of success. Leveraging these findings, along with recent evidence of the striking efficacy of the national network of university-based “Agriculture Experiment Stations” established by the Hatch Act of 1887, we propose a national network of university-based “Policy Experiment Stations” or policy innovation labs in each state, supported by continuing federal and state appropriations and tasked with accelerating state and local government innovation.  

Challenge

Advocates of abundance have identified “failed public policy” as an increasingly significant barrier to economic growth and community flourishing. Of particular concern are state and local policies and programs, including those powered by federal funds, that do not effectively deliver critically important public goods and services like health, education, safety, clean air and water, and growth-oriented infrastructure.

Part of the challenge is that state and local governments lack capacity to conduct rigorous evaluations of the efficacy of their policies and programs. For example, the American Rescue Plan, the largest one-time federal investment in state and local governments in the last century, provided $350 billion in State and Local Fiscal Recovery Funds to state, territorial, local, and Tribal governments to accelerate post-pandemic economic recovery. Yet very few of those investments are being evaluated for efficacy. In a recent survey of state policymakers, 59% of those surveyed cited “lack of time for rigorous evaluations” as a key obstacle to innovation. State and local governments also typically lack the time, resources, and technical capacity to canvass evaluation evidence from other settings and assess whether a program proven to improve outcomes elsewhere might also improve outcomes locally. Finally, state and local governments often don’t adopt more effective programs even when they have rigorous evidence that these programs are more effective than the status quo, because implementing new programs disrupts existing workflows. 

If state and local policymakers don’t know what works and what doesn’t, and/or aren’t able to overcome even relatively minor implementation challenges when they do know what works, they won’t be able to spend federal dollars more effectively, or more generally to deliver critical public goods and services.

Opportunity

A growing body of research on government innovation is documenting factors that reliably increase the likelihood that governments will implement evidence-based policy reform. First, government decision makers are more likely to adopt evidence-based policy reforms when they are grounded in local evidence and/or recommended by local researchers. Boston-based researchers sharing a Boston-based study showing that relaxing density restrictions reduces rents and house prices will do less to convince San Francisco decision makers than either a San Francisco-based study, or San Francisco-based researchers endorsing the evidence from Boston. Proximity matters for government innovation.

Second, government decision makers are more likely to adopt evidence-based policy reforms when they are engaged as partners in the research projects that produce the evidence of efficacy, helping to define the set of feasible policy alternatives and design new policy interventions. Research partnerships matter for government innovation.

Third, evidence-based policies are significantly more likely to be adopted when the policy innovation is part of an existing implementation infrastructure, or when agencies receive dedicated implementation support. This means that moving beyond incremental policy reforms will require that state and local governments receive more technical support in overcoming implementation challenges. Implementation matters for government innovation. 

We know that the implementation of evidence-based policy reform produces returns for communities that have been estimated to be on the order of 17:1. Our partners in government have voiced their direct experience of these returns. In Puerto Rico, for example, decision makers in the Department of Education have attributed the success of evidence-based efforts to help students learn to the “constant communication and effective collaboration” with researchers who possessed a “strong understanding of the culture and social behavior of the government and people of Puerto Rico.” Carrie S. Cihak, the evidence and impact officer for King County, Washington, likewise observes, 

“It is critical to understand whether the programs we’re implementing are actually making a difference in the communities we serve. Throughout my career in King County, I’ve worked with  County teams and researchers on evaluations across multiple policy areas, including transportation access, housing stability, and climate change. Working in close partnership with researchers has guided our policymaking related to individual projects, identified the next set of questions for continual learning, and has enabled us to better apply existing knowledge from other contexts to our own. In this work, it is essential to have researchers who are committed to valuing local knowledge and experience–including that of the community and government staff–as a central part of their research, and who are committed to supporting us in getting better outcomes for our communities.” 

The emerging body of evidence on the determinants of government innovation can help us define a plan of action that galvanizes the state and local government innovation necessary to accelerate regional economic growth and community flourishing. 

Plan of Action 

An evidence-based plan to increase state and local government innovation needs to facilitate and sustain durable partnerships between state and local governments and neighboring universities to produce scientifically rigorous policy evaluations, adapt evaluation evidence from other contexts, and develop effective implementation strategies. Over a century ago, the Hatch Act of 1887 created a remarkably effective and durable R&D infrastructure aimed at agricultural innovation, establishing university-based Agricultural Experiment Stations (AES) in each state tasked with developing, testing, and translating innovations designed to increase agricultural productivity. 

Locating university-based AES in every state ensured the production and implementation of locally-relevant evidence by researchers working in partnership with local stakeholders. Federal oversight of the state AES by an Office of Experiment Stations in the US Department of Agriculture ensured that work was conducted with scientific rigor and that local evidence was shared across sites. Finally, providing stable annual federal appropriations for the AES, with required matching state appropriations, ensured the durability and financial sustainability of the R&D infrastructure. This infrastructure worked: agricultural productivity near the experiment stations increased by 6% after the stations were established.

Congress should develop new legislation to create and fund a network of state-based “Policy Experiment Stations.”

 The 119th Congress that will convene on January 3, 2025 can adapt the core elements of the proven-effective network of state-based Agricultural Experiment Stations to accelerate state and local government innovation. Mimicking the structure of 7 USC 14, federal grants to states would support university-based “Policy Experiment Stations” or policy innovation labs in each state, tasked with partnering with state and local governments on (1) scientifically rigorous evaluations of the efficacy of state and local policies and programs; (2) translations of evaluation evidence from other settings; and (3) overcoming implementation challenges. 

As in 7 USC 14, grants to support state policy innovation labs would be overseen by a federal office charged with ensuring that work was conducted with scientific rigor and that local evidence was shared across sites. We see two potential paths for this oversight function, paths that in turn would influence legislative strategy.

Pathway 1: This oversight function could be located in the Office of Evaluation Sciences (OES) in the General Services Administration (GSA). In this case, the congressional committees overseeing GSA, namely the House Committee on Oversight and Responsibility and the Senate Committee on Homeland Security and Governmental Affairs, would craft legislation providing for an appropriation to GSA to support a new OES grants program for university-based policy innovation labs in each state. The advantage of this structure is that OES is a highly respected locus of program and policy evaluation expertise. 

Pathway 2: Oversight could instead be located in the Directorate of Technology, Innovation, and Partnerships in the National Science Foundation (NSF TIP). In this case, the House Committee on Science, Space, and Technology and the Senate Committee on Commerce, Science, and Transportation would craft legislation providing for a new grants program within NSF TIP to support university-based policy innovation labs in each state. The advantage of this structure is that NSF is a highly respected grant-making agency. 

Either of these paths is feasible with bipartisan political will. Alternatively, there are unilateral steps that could be taken by the incoming administration to advance state and local government innovation. For example, the Office of Management and Budget (OMB) recently released updated Uniform Grants Guidance clarifying that federal grants may be used to support recipients’ evaluation costs, including “conducting evaluations, sharing evaluation results, and other personnel or materials costs related to the effective building and use of evidence and evaluation for program design, administration, or improvement.” The Uniform Grants Guidance also requires federal agencies to assess the performance of grant recipients, and further allows federal agencies to require that recipients use federal grant funds to conduct program evaluations. The incoming administration could further update the Uniform Grants Guidance to direct federal agencies to require that state and local government grant recipients set aside grant funds for impact evaluations of the efficacy of any programs supported by federal funds, and further clarify the allowability of subgrants to universities to support these impact evaluations.

Conclusion

Establishing a national network of university-based “Policy Experiment Stations” or policy innovation labs in each state, supported by continuing federal and state appropriations, is an evidence-based plan to facilitate abundance-oriented state and local government innovation. We already have impressive examples of what these policy labs might be able to accomplish. At MIT’s Abdul Latif Jameel Poverty Action Lab North America, the University of Chicago’s Crime Lab and Education Lab, the University of California’s California Policy Lab, and Harvard University’s The People Lab, to name just a few, leading researchers partner with state and local governments on scientifically rigorous evaluations of the efficacy of public policies and programs, the translation of evidence from other settings, and overcoming implementation challenges, leading in several cases to evidence-based policy reform. Yet effective as these initiatives are, they are largely supported by philanthropic funds, an infeasible strategy for national scaling.

In recent years we’ve made massive investments in communities through federal grants to state and local governments. We’ve also initiated ambitious efforts at growth-oriented regulatory reform which require not only federal but also state and local action. Now it’s time to invest in building state and local capacity to deploy federal investments effectively and to galvanize regional economic growth. Emerging research findings about the determinants of government innovation, and about the efficacy of the R&D infrastructure for agricultural innovation established over a century ago, give us an evidence-based roadmap for state and local government innovation.

This action-ready policy memo is part of Day One 2025 — our effort to bring forward bold policy ideas, grounded in science and evidence, that can tackle the country’s biggest challenges and bring us closer to the prosperous, equitable and safe future that we all hope for whoever takes office in 2025 and beyond.

Critical minerals, robotics, advanced energy systems, quantum computing, biotechnology, shipbuilding, and space are some of the resources and technologies that will define the economic and security climate of the 21st century. However, the United States is at risk of losing its edge in these technologies of the future. For instance, China processes the vast majority of the world’s batteries and critical metals and has successfully launched a quantum communications satellite. The implications are enormous: the U.S. relies on its qualitative technological edge to fuel productivity growth, improve living standards, and maintain the existing global order. Indeed, the Inflation Reduction Act (IRA) and CHIPS Act were largely reactionary moves to shore up atrophied manufacturing capabilities in the American battery and semiconductor industries, requiring hundreds of billions in outlays to catch up. In an ideal world, critical industries would be sufficiently funded well in advance to avoid economically costly catch-up spending.

However, many of these technologies are characterized by long timelines, significant capital expenditures, and low and uncertain profit margins, presenting major challenges for private-sector investors who are required by their limited partners (capital providers such as pension funds, university endowments, and insurance companies) to underwrite to a certain risk-adjusted return threshold. This stands in contrast to technologies like artificial intelligence and pharmaceuticals: While both are also characterized by large upfront investments and lengthy research and development timelines, the financial payoffs are far clearer, incentivizing private sectors to play a leading role in commercialization. This issue for technologies in economically and geopolitically vital industries such as lithium processing and chips is most acute in the “valley of death,” when companies require scale-up capital for an early commercialization effort: the capital required is too large for traditional venture capital, yet too risky for traditional project finance.

The United States needs a strategic investment fund (SIF) to shepherd promising technologies in nationally vital sectors through the valley of death. An American SIF is not intended to provide subsidies, pick political winners or losers, or subvert the role of private capital markets. On the contrary, its role would be to “crowd in” capital by uniquely managing risk that no private or philanthropic entities have the capacity to do. In doing so, an SIF would ensure that the U.S. maintains an edge in critical technologies, promoting economic dynamism and national security in an agile, cost-efficient manner. 

Challenges

The Need for Private Investment 

A handful of resources and technologies, some of which have yet to be fully characterized, have the potential to play an outsized role in the future economy. Most of these key technologies have meaningful national security implications.

Since ChatGPT’s release in November 2022, artificial intelligence (AI) has experienced a commercial renaissance that has captured the public’s imagination and huge sums of venture dollars, as evidenced by OpenAI’s October 2024 $6.5 billion round at a $150 billion pre-money valuation. However, AI is not the only critical resource or technology that will power the future economy, and many of those critical resources and technologies may struggle to attract the same level of private investment. Consider the following:

• To meet climate goals, the world needs to increase production of lithium by nearly 475%, rare earths by 100%, and nickel by 60% through 2035. For defense applications, rare earths are especially important; the construction of one F-35, for instance, uses 920 pounds of rare earth materials.
• The conflict in Ukraine has unequivocally demonstrated the value of low-cost drones on the battlefield. However, drones also have significant commercial applications, including safety and last-mile delivery. Reducing production and component costs could make a meaningful difference.
• Quantum technology has the potential to exponentially expand compute power, which can be used to simulate biological pathways, accelerate materials development, and process vast amounts of financial data. However, quantum technology can also be used to break existing encryption technologies and safeguard communications. China launched its first quantum communications satellite in 2020.

Few sectors receive the level of consistent venture attention that software technology, most recently in AI, has gotten in the last 18 months. However, this does not make them unbackable or unimportant; on the contrary, technologies that increase mineral recovery yields or make drone engines cheaper should receive sufficient support to get to scale. While private-sector capital markets have supported the development of many important industries, they are not perfect and may miss important opportunities due to information asymmetries and externalities.

Overcoming the Valley of Death

Many strategically important technologies are characterized by high upfront costs and low or uncertain margins, which tends to dissuade investment by private-sector organizations at key inflection points, namely, the “valley of death.”

By their nature, innovative technologies are complex and highly uncertain. However, some factors make future economic value—and therefore financeability—more difficult to ascertain than others. For example, innovative battery technologies that enable long-term storage of energy generated from renewables would greatly improve the economics of utility-scale solar and wind projects. However, this requires production at scale in the face of potential competition from low-cost incumbents. In addition, there is the element of scientific risk itself, as well as the question of customer adoption and integration. There are many good reasons why technologies and companies that seem feasible, economical, and societally valuable do not succeed.

These dynamics result in lopsided investment allocations. In the early stages of innovation, venture capital is available to fund startups with the promise of outsized return driven partially by technological hype and partially by the opportunity to take large equity stakes in young companies. At the other end of the barbell, private equity and infrastructure capital are available to mature companies seeking an acquisition or project financing based on predictable cash flows and known technologies. 

However, gaps appear in the middle as capital requirements increase (often by an order of magnitude) to support the transition to early commercialization. This phenomenon is called the “valley of death” as companies struggle to raise the capital they need to get to scale given the uncertainties they face.

Figure 1. The “valley of death” describes the mismatch between existing financial structures and capital requirements in the crucial early commercialization phase. (Source: Maryland Energy Innovation Accelerator)

Shortcoming of Federal Subsidies

While the federal government has provided loans and subsidies in the past, its programs remain highly reactive and require large amounts of funding.

Aside from asking investors to take on greater risk and lower returns, there are several tools in place to ameliorate the valley of death. The IRA one such example: It appropriated some $370 billion for climate-related spending with a range of instruments, including tax subsidies for renewable energy production, low-cost loans through organizations such as the Department of Energy’s Loan Program Office (LPO), and discretionary grants.

On the other hand, there are major issues with this approach. First, funding is spread out across many calls for funding that tend to be slow, opaque, and costly. Indeed, it is difficult to keep track of available resources, funding announcements, and key requirements—just try searching for a comprehensive, easy-to-understand list of opportunities.

More importantly, these funding mechanisms are simply expensive. The U.S. does not have the financial capacity to support an IRA or CHIPS Act for every industry, nor should it go down that route. While one could argue that these bills reflect the true cost of achieving the stated policy aims of energy transition or securing the semiconductor supply chain, it is also the case that there both knowledge (engineering expertise) and capital (manufacturing facility) capabilities underpin these technologies. Allowing these networks to atrophy created greater costs down the road, which could have been prevented by targeted investments at the right points of development.

The Future Is Dynamic

The future is not perfectly knowable, and new technological needs may arise that change priorities or solve previous problems. Therefore, agility and constant re-evaluation are essential.

Technological progress is not static. Take the concept of peak oil: For decades, many of the world’s most intelligent geologists and energy forecasters believed that the world would quickly run out of oil reserves as the easiest to extract resources were extracted. In reality, technological advances in chemistry, surveying, and drilling enabled hydraulic fracturing (fracking) and horizontal drilling, creating access to “unconventional reserves” that substantially increased fossil fuel supply.

Figure 2. In 1956, M.K. Hubbert created “peak oil” theory, projecting that reserves would be exhausted around the turn of the millennium.

Fracking greatly expanded fossil fuel production in the U.S., increasing resource supply, securing greater energy independence, and facilitating the transition from coal to natural gas, whose expansion has proved to be a helpful bridge towards renewable energy generation. This transition would not have been possible without a series of technological innovations—and highly motivated entrepreneurs—that arose to meet the challenge of energy costs.

To meet the challenges of tomorrow, policymakers need tools that provide them with flexible and targeted options as well as sufficient scale to make an impact on technologies that might need to get through the valley of death. However, they need to remain sufficiently agile so as not to distort well-functioning market forces. This balance is challenging to achieve and requires an organizational structure, authorizations, and funding mechanisms that are sufficiently nimble to adapt to changing technologies and markets.

Opportunity

Given these challenges, it seems unlikely that solutions that rely solely on the private sector will bridge the commercialization gap in a number of capital-intensive strategic industries. On the other hand, existing public-sector tools, such as grants and subsidies, are too costly to implement at scale for every possible externality and are generally too retrospective in nature rather than forward-looking. The government can be an impactful player in bridging the innovation gap, but it needs to do so cost-efficiently.

An SIF is a promising potential solution to the challenges posed above. By its nature, an SIF would have a public mission focused on strategic technologies crossing the valley of death by using targeted interventions and creative financing structures that crowd in private investors. This would enable the government to more sustainably fund innovation, maintain a light touch on private companies, and support key industries and technologies that will define the future global economic and security outlook.

Plan of Action

Recommendation 1. Shepherd technologies through the valley of death. 

While the SIF’s investment managers are expected to make the best possible returns, this is secondary to the overarching public policy goal of ensuring that strategically and economically vital technologies have an opportunity to get to commercial scale.

The SIF is meant to crowd in capital such that we achieve broader societal gains—and eventually, market-rate returns—enabled by technologies that would not have survived without timely and well-structured funding. This creates tension between two competing goals: The SIF needs to act as if it will intend to make returns, or else there is the potential for moral hazard and complacency. However, it also has to be willing to not make market-rate returns, or even lose some of its principal, in the service of broader market and ecosystem development. 

Thus, it needs to be made explicitly clear from the beginning that an SIF has the intent of achieving market rate returns by catalyzing strategic industries but is not mandated to do so. One way to do this is to adopt a 501(c)(3) structure that has a loose affiliation to a department or agency, similar to that of In-Q-Tel. Excess returns could either be recycled to the fund or distributed to taxpayers.

The SIF should adapt the practices, structures, and procedures of established private-sector funds. It should have a standing investment committee made up of senior stakeholders across various agencies and departments (expanded upon below). Its day-to-day operations should be conducted by professionals who provide a range of experiences, including investing, engineering and technology, and public policy across a spectrum of issue areas. 

In addition, the SIF should develop clear underwriting criteria and outputs for each investment. These include, but are not limited to, identifying the broader market and investment thesis, projecting product penetration, and developing potential return scenarios based on different permutations of outcomes. More critically, each investment needs to create a compelling case for why the private sector cannot fund commercialization on its own and why public catalytic funding is essential.

Recommendation 2. The SIF should have a permanent authorization to support innovation under the Department of Commerce. 

The SIF should be affiliated with the Department of Commerce but work closely with other departments and agencies, including the Department of Energy, Department of Treasury, Department of Defense, Department of Health and Human Services, National Science Foundation, and National Economic Council.

Strategic technologies do not fall neatly into one sector and cut across many customers. Siloing funding in different departments misses the opportunity to capture funding synergies and, more importantly, develop priorities that are built through information sharing and consensus. Enter the Department of Commerce. In addition to administering the National Institute of Standards and Technology, they have a strong history of working across agencies, such as with the CHIPS Act.

Similar arguments can also be made for the Treasury, and it may even be possible to have Treasury and Commerce work together to manage an SIF. They would be responsible for bringing in subject matter experts (for example, from the Department of Energy or National Science Foundation) to provide specific inputs and arguments for why specific technologies need government-based commercialization funding and at what point such funding is appropriate, acting as an honest broker to allocate strategic capital.

To be clear: The SIF is not intended to supersede any existing funding programs (e.g., the Department of Energy’s Loan Program Office or the National Institute of Health’s ARPA-H) that provide fit-for-purpose funding to specific sectors. Rather, an SIF is intended to fill in the gaps and coordinate with existing programs while providing more creative financing structures than are typically available from government programs.

Recommendation 3. Create a clear innovation roadmap.

Every two years, the SIF should develop or update a roadmap of strategically important industries, working closely with private, nonprofit, and academic experts to define key technological and capability gaps that merit public sector investment. 

The SIF’s leaders should be empowered to make decisions on areas to prioritize but have the ability to change and adapt as the economic environment evolves. Although there is a long list of industries that an SIF could potentially support, resources are not infinite. However, a critical mass of investment is required to ensure adequate resourcing. One acute challenge is that this is not perfectly known in advance and changes depending on the technology and sector. However, this is precisely what the strategic investment roadmap is supposed to solve for: It should provide an even-handed assessment of the likely capital requirements and where the SIF is best suited to provide funding compared to other agencies or the private sector.

Moreover, given the ever-changing nature of technology, the SIF should frequently reassess its understanding of key use cases and their broader economic and strategic importance. Thus, after initial development of the SIF, it should be updated every two years to ensure that its takeaways and priorities remain relevant. This is no different than documents such as the National Security Strategy, which are updated every two to four years; in fact, the SIF’s planning documents should flow seamlessly into the National Security Strategy. 

To provide a sufficiently broad set of perspectives, the government should include the expertise and insights of outside experts to develop its plan. Existing bodies, such as the President’s Council of Advisors on Science and Technology and the National Quantum Initiative, provide some of the consultative expertise required. However, the SIF should also stand up subject matter specific advisory bodies where a need arises (for example, on critical minerals and mining) and work internally to set specific investment areas and priorities.

Recommendation 4. Limit the SIF to financing.

The government should not be an outsized player in capital markets. As such, the SIF should receive no governance rights (e.g., voting or board seats) in the companies that it invests in.

Although the SIF aims to catalyze technological and ecosystem development, it should be careful not to dictate the future of specific companies. Thus, the SIF should avoid information rights beyond financial reporting. Typical board decks and stockholder updates include updates on customers, technologies, personnel matters, and other highly confidential and specific pieces of information that, if made public through government channels, would play a highly distortionary role in markets. Given that the SIF is primarily focused on supporting innovation through a particularly tricky stage to navigate, the SIF should receive the least amount of information possible to avoid disrupting markets.

Recommendation 5. Focus on providing first-loss capital.

First-loss capital should be the primary mechanism by which the SIF supports new technologies, providing greater incentives for private-sector funders to support early commercialization while providing a means for taxpayers to directly participate in the economic upside of SIF-supported technologies.

Consider the following stylized example to demonstrate a key issue in the valley of death. A promising clean technology company, such as a carbon-free cement or long-duration energy storage firm, is raising $100mm of capital for facility expansion and first commercial deployment. To date, the company has likely raised $30 – $50mm of venture capital to enable tech development, pilot the product, and grow the team’s engineering, R&D, and sales departments.

However, this company faces a fundraising dilemma. Its funding requirements are now too big for all but the largest venture capital firms, who may or may not want to invest in projects and companies like these. On the other hand, this hypothetical company is not mature enough for private equity buyouts nor is it a good candidate for typical project-based debt, which typically require several commercial proof points in order to provide sufficient risk reduction for an investor whose upside is relatively limited. Hence, the “valley of death.”

First-loss capital is an elegant solution to this issue: A prospective funder could commit to equal (pro rata) terms as other investors, except that this first-loss funder is willing to use its investment to make other investors whole (or at least partially offset losses) in the event that the project or company does not succeed. In this example, a first-loss funder would commit to $33.5 million of equity funding (roughly one-third of the company’s capital requirement). If the company succeeds, the first-loss funder makes the same returns as the other investors. However, if the company is unable to fully meet these obligations, the first-loss funder’s $33.5 million would be used to pay the other investors back (the other $66.5 million that was committed). This creates a floor on losses for the non-first-loss investors: Rather than being at risk of losing 100% of their principal, they are at risk of losing 50% of their principal.

The creation of a first-loss layer has a meaningful impact on the risk-reward profile for non-first-loss investors, who now have a floor on returns (in the case above, half their investment). By expanding the acceptable potential loss ratio, growth equity capital (or another appropriate instrument, such as project finance) can fill the rest, thereby crowding in capital. 

From a risk-adjusted returns standpoint, this is not a free lunch for the government or taxpayers. Rather, it is intended to be a capital-efficient way of supporting the private-sector ecosystem in developing strategically and economically vital technologies. In other words, it leverages the power of the private sector to solve externalities while providing just enough support to get them to the starting line in the first place.

Conclusion

Many of tomorrow’s strategically important technologies face critical funding challenges in the valley of death. Due to their capital intensity and uncertain outcomes, existing financing tools are largely falling short in the critical early commercialization phases. However, a nimble, properly funded SIF could bridge key gaps while allowing the private sector to do most of the heavy lifting. The SIF would require buy-in from many stakeholders and well-defined sources of funding, but these can be solved with the right mandates, structures, and pay-fors. Indeed, the stakes are too high, and the consequences too dire, to not get strategic innovation right in the 21st century.

This action-ready policy memo is part of Day One 2025 — our effort to bring forward bold policy ideas, grounded in science and evidence, that can tackle the country’s biggest challenges and bring us closer to the prosperous, equitable and safe future that we all hope for whoever takes office in 2025 and beyond.

Frequently Asked Questions

If first-loss capital is such a powerful mechanism, why doesn’t the market already provide it?

Put simply, there needs to be an entity that is actually willing and able to absorb lower returns, or even lose some of its principal, in the service of building an ecosystem. Even if the “median” outcome is a market-rate return of capital, the risk-adjusted returns are in effect far lower because the probability of a zero outcome for first-loss providers is substantially nonzero. Moreover, it’s not clear exactly what the right probability estimate should be; therefore, it requires a leap of faith that no economically self-interested private market actor would be willing to take. While some quasi-social-sector organizations can play this role (for example, Bill Gates’s Breakthrough Energy Ventures for climate tech), their capacity is finite, nor is there a guarantee that such a vehicle will appear for every sector of interest. Therefore, a publicly funded SIF is an integral solution to bridging the valley of death.

read more

Does the SIF always have to use first-loss structures?

No, the SIF would not always have to use first-loss structures. However, it is the most differentiated structure that is available to the U.S. government; otherwise, a private-sector player is likely able—and better positioned—to provide funding.

read more

What type of financial instruments (e.g., debt, equity) would the SIF use?

The SIF should be able to use the full range of instruments, including project finance, corporate debt, convertible loans, and equity capital, and all combinations thereof. The instrument of choice should be up to the judgment of the applicant and SIF investment team. This is distinct from providing first-loss capital: Regardless of the financial instrument used, the SIF’s investment would be used to buffer other investors against potential losses.

read more

What is the target rate of return?

The target return rate should be commensurate with that of the instrument used. For example, mezzanine debt should target 13–18% IRR, while equity investments should aim for 20–25% IRR. However, because of the increased risk of capital loss to the SIF given its first loss position, the effective blended return should be expected to be lower. 

The SIF should be prepared to lose capital on each individual investment and, as a blended portfolio, to have negative returns. While it should underwrite such that it will achieve market-rate returns if successful in crowding in other capital that improves the commercial prospects of technologies and companies in the valley of death, the SIF has a public goal of ecosystem development for strategic domains. Therefore, lower-than-market-rate returns, and even some principal degradation, is acceptable but should be avoided as much as possible through the prudence of the investment committee.

read more

What protections will the SIF be offered, if they do not get board seats or representation?

By and large, the necessary public protections are granted through CFIUS, which requires regulatory approval for export or foreign ownership stakes with voting rights above 25% of critical technologies. The SIF can also enact controls around information rights (e.g., customer lists, revenue, product roadmaps) such that they have a veto on parties that can receive such information. However, given its catalytic mission, the SIF does not need board seats or representation and should focus on ensuring that critical technologies and assets are properly protected.

read more

What is the overall investment committee composition, and what are the processes for approval?

In most private investment firms, the investment committee is made up of the most senior individuals in the fund. These individuals can cross asset classes, sectors of expertise, and even functional backgrounds. However, the investment committee represents a wide breadth of expertise and experiences that, when brought together, enable intellectual honesty and the application of collective wisdom and judgment to the opportunity at hand.

Similarly, the SIF’s investment committee could include the head of the fund and representatives from various departments and agencies in alignment with its strategic priorities. The exact size of the investment committee should be defined by these priorities, but approval should be driven by consensus, and unanimity (or near unanimity) should be expected for investments that are approved.

Given the fluid nature of investment opportunities, the committee should be called upon whenever needed to evaluate a potential opportunity. However, given the generally long process times for investments discussed above (6–12 months), the investment committee should have been briefed multiple times before a formal decision is made.

read more

What check size is envisioned? Is there a strict cutoff for the amount of funding that can be disbursed?

Check sizes can be flexible to the needs of the investment opportunity. However, as an initial guiding principle, first loss capital should likely make up 20–35% of capital invested so as to require private-sector investors to have meaningful skin in the game. Depending on the fundraise size, this could imply investments of $25 million to $100 million.

Target funding amounts should be set over multiyear timeframes, but the annual appropriations process implies that there will likely be a set cap in any given year. In order to meet the needs of the market, there should be mechanisms that enable emergency draws, up to a cap (e.g., 10% of the annual target funding amount, which will need to be “paid for” by reducing future outlays). 

read more

How will the SIF be funded?

An economically efficient way to fund a government program in support of a positive externality is a Pigouvian tax on negative externalities (such as carbon). However, carbon taxes are as politically unappealing as they are economically sensible and need to be packaged into other policy goals that could potentially support such legislation. Notwithstanding the questionable economic wisdom of tariffs in general, some 56% of voters support a 10% tax on all imports and 60% tariffs on China. Rather than using tariffs harmfully, they could be used more productively. One such proposal is a carbon import tariff that taxes imports on the carbon emitted in the production and transportation of goods into the U.S.

The U.S. would not be a first mover: in fact, the European Union has already implemented a similar mechanism called the Carbon Border Adjustment Mechanism (CBAM), which is focused on heavy industry, including cement, iron and steel, aluminum, fertilizers, electricity, and hydrogen, with chemicals and polymers potentially to be included after 2026. At full rollout in 2030, the CBAM is expected to generate roughly €10–15 billion of tax revenue. Tax receipts of a similar size could be used to fund an SIF or, if Congress authorizes an upfront amount, could be used to nullify the incremental deficit over time.

read more

If the carbon innovation fee is implemented, how would levies be assessed and exemptions provided?

The EU’s CBAM phased in its reporting requirements over several years. Through July 2024, companies were allowed to use default amounts per unit of production without an explanation as to why actual data was not used. Until January 1, 2026, companies can make estimates for up to 20% of goods; thereafter, the CBAM requires reporting of actual quantities and embedded greenhouse gas emissions.

The U.S. could use a similar phase-in, although given the challenges of carbon reporting, could allow companies to use the lower of actual, verified emissions or per-unit estimates. Under a carbon innovation fee regime, exporters and countries could apply for exemption on a case-by-case basis to the Department of Commerce, which they could approve in line with other goals (e.g., economic development in a region).

read more

Besides a carbon innovation fee, what are other ways to fund the SIF?

The SIF could also be funded by repurposing other funding and elevating their strategic importance. Potential candidates include the Small Business Innovation Research (SBIR) and Small State Business Credit Initiative (SSBCI), which could play a bigger role if moved into the SIF umbrella. For example, the SBIR program, whose latest reporting data is as of FY2019, awarded $3.3 billion in funding that year and $54.6 billion over its lifespan. Moreover, the SSBCI, a $10 billion fund that already provides loan guarantees and other instruments similar to those described above, can be used to support technologies that fall into the purview of the SIF.

read more

What funding options are available if reallocation of existing funds isn’t an option?

Congress could also assess reallocating dollars towards an SIF from spending reforms that are likely inevitable given the country’s fiscal position. In 2023, the Congressional Budget Office (CBO) published a report highlighting potential solutions for reducing the budget deficit. Some potential solutions, like establishing caps on Medicaid federal spending, while fiscally promising, seem unlikely to pass in the near future. However, others are more palatable, especially those that eliminate loopholes or ask higher-income individuals to pay their fair share.

For instance, increasing the amount subject to Social Security taxes above the $250,000 threshold has the potential to raise up to $1.2 trillion over 10 years; while this can be calibrated, an SIF would take only a small fraction of the taxes raised. In addition, the CBO found that federal matching funds for Medicaid frequently ended up getting back to healthcare providers in the form of higher reimbursement rates; eliminating what are effectively kickbacks could reduce the deficit by up to $525 billion over 10 years.

read more

There is a crisis within healthcare technology research and development, wherein certain groups due to their age, gender, or race and ethnicity are under-researched in preclinical studies, under-represented in clinical trials, misunderstood by clinical practitioners, and harmed by biased medical technology. These issues in turn contribute to costly disparities in healthcare outcomes, leading to losses of $93 billion a year in excess medical-care costs, $42 billion a year in lost productivity, and $175 billion a year due to premature deaths. With the rise of artificial intelligence (AI) in healthcare, there’s a risk of encoding and recreating existing biases at scale.

The next Administration and Congress must act to address bias in medical technology at the development, testing and regulation, and market-deployment and evaluation phases. This will require coordinated effort across multiple agencies. In the development phase, science funding agencies should enforce mandatory subgroup analysis for diverse populations, expand funding for under-resourced research areas, and deploy targeted market-shaping mechanisms to incentivize fair technology. In the testing and regulation phase, the FDA should raise the threshold for evaluation of medical technologies and algorithms and expand data-auditing processes. In the market-deployment and evaluation phases, infrastructure should be developed to perform impact assessments of deployed technologies and government procurement should incentivize technologies that improve health outcomes.

Challenge and Opportunity

Bias is regrettably endemic in medical innovation. Drugs are incorrectly dosed to people assigned female at birth due to historical exclusion of women from clinical trials. Medical algorithms make healthcare decisions based on biased health data, clinically disputed race-based corrections, and/or model choices that exacerbate healthcare disparities. Much medical equipment is not accessible, thus violating the Americans with Disabilities Act. And drugs, devices, and algorithms are not designed with the lifespan in mind, impacting both children and the elderly. Biased studies, technology, and equipment inevitably produce disparate outcomes in U.S. healthcare.

The problem of bias in medical innovation manifests in multiple ways: cutting across technological sectors in clinical trials, pervading the commercialization pipeline, and impeding equitable access to critical healthcare advances.

Bias in medical innovation starts with clinical research and trials

The 1993 National Institutes of Health (NIH) Revitalization Act required federally funded clinical studies to (i) include women and racial minorities as participants, and (ii) break down results by sex and race or ethnicity. As of 2019, the NIH also requires inclusion of participants across the lifespan, including children and older adults. Yet a 2019 study found that only 13.4% of NIH-funded trials performed the mandatory subgroup analysis, and challenges in meeting diversity targets continue into 2024 . Moreover, the increasing share of industry-funded studies are not subject to Revitalization Act mandates for subgroup analysis. These studies frequently fail to report differences in outcomes by patient population as a result. New requirements for Diversity Action Plans (DAPs), mandated under the 2023 Food and Drug Omnibus Reform Act, will ensure drug and device sponsors think about enrollment of diverse populations in clinical trials. Yet, the FDA can still approve drugs and devices that are not in compliance with their proposed DAPs, raising questions around weak enforcement. 

The resulting disparities in clinical-trial representation are stark: African Americans represent 12% of the U.S. population but only 5% of clinical-trial participants, Hispanics make up 16% of the population but only 1% of clinical trial participants, and sex distribution in some trials is 67% male. Finally, many medical technologies approved prior to 1993 have not been reassessed for potential bias. One outcome of such inequitable representation is evident in drug dosing protocols: sex-aware prescribing guidelines exist for only a third of all drugs.

Bias in medical innovation is further perpetuated by weak regulation

Algorithms

Regulation of medical algorithms varies based on end application, as defined in the 21^st Century Cures Act. Only algorithms that (i) acquire and analyze medical data and (ii) could have adverse outcomes are subject to FDA regulation. Thus, clinical decision-support software (CDS) is not regulated even though these technologies make important clinical decisions in 90% of U.S. hospitals. The FDA has taken steps to try and clarify what CDS must be considered a medical device, although these actions have been heavily criticized by industry. Finally, the lack of regulatory frameworks for generative AI tools is leading to proliferation without oversight.

Even when a medical algorithm is regulated, regulation may occur through relatively permissive de novo pathways and 510(k) pathways. A de novo pathway is used for novel devices determined to be low to moderate risk, and thus subject to a lower burden of proof with respect to safety and equity. A 510(k) pathway can be used to approve a medical device exhibiting “substantial equivalence” to a previously approved device, i.e., it has the same intended use and/or same technological features. Different technical features can be approved so long as there are no questions raised around safety and effectiveness.

Medical algorithms approved through de novo pathways can be used as predicates for approval of devices through 510(k) pathways. Moreover, a device approved through a 510(k) pathway can remain on the market even if its predicate device was recalled. Widespread use of 510(k) approval pathways has generated a “collapsing building” phenomenon, wherein many technologies currently in use are based on failed predecessors. Indeed, 97% of devices recalled between 2008 to 2017 were approved via 510(k) clearance. 

While DAP implementation will likely improve these numbers, for the 692 AI-ML enabled medical devices, only 3.6% reported race or ethnicity, 18.4% reported age, and only .9% include any socioeconomic information. Further, less than half did detailed analysis of algorithmic performance and only 9% included information on post-market studies, raising the risk of algorithmic bias following approvals and broad commercialization.

Even more alarming is evidence showing that machine learning can further entrench medical inequities. Because machine learning medical algorithms are powered by data from past medical decision-making, which is rife with human error, these algorithms can perpetuate racial, gender, and economic bias. Even algorithms demonstrated to be ‘unbiased’ at the time of approval can evolve in biased ways over time, with little to no oversight from the FDA. As technological innovation progresses, especially generative AI tools, an intentional focus on this problem will be required.

Medical devices

Currently, the Medical Device User Fee Act requires the FDA to consider the least burdensome appropriate means for manufacturers to demonstrate the effectiveness of a medical device or to demonstrate a device’s substantial equivalence. This requirement was reinforced by the 21^st Century Cures Act, which also designated a category for “breakthrough devices” subject to far less-stringent data requirements. Such legislation shifts the burden of clinical data collection to physicians and researchers, who might discover bias years after FDA approval. This legislation also makes it difficult to require assessments on the differential impacts of technology.

Like medical algorithms, many medical devices are approved through 510(k) exemptions or de novo pathways. The FDA has taken steps since 2018 to increase requirements for 510(k) approval and ensure that Class III (high-risk) medical devices are subject to rigorous pre-market approval, but problems posed by equivalence and limited diversity requirements remain. 

Finally, while DAPs will be required for many devices seeking FDA approval, the recommended number of patients in device testing is shockingly low. For example, currently, only 10 people are required in a study of any new pulse oximeter’s efficacy and only 2 of those people need to be “darkly pigmented”. This requirement (i) does not have the statistical power necessary to detect differences between demographic groups, and (i) does not represent the composition of the U.S. population. The standard is currently under revision after immense external pressure. FDA-wide, there are no recommended guidelines for addressing human differences in device design, such as pigmentation, body size, age, and pre-existing conditions.

Pharmaceuticals

The 1993 Revitalization Act strictly governs clinical trials for pharmaceuticals and does not make recommendations for adequate sex or genetic diversity in preclinical research. The results are that a disproportionately high number of male animals are used in research and that only 5% of cell lines used for pharmaceutical research are of African descent. Programs like All of Us, an effort to build diverse health databases through data collection, are promising steps towards improving equity and representation in pharmaceutical research and development (R&D). But stronger enforcement is needed to ensure that preclinical data (which informs function in clinical trials) reflects the diversity of our nation. 

Bias in medical innovation are not tracked post-regulatory approval

FDA-regulated medical technologies appear trustworthy to clinicians, where the approval signals safety and effectiveness. So, when errors or biases occur (if they are even noticed), the practitioner may blame the patient for their lifestyle rather than the technology used for assessment. This in turn leads to worse clinical outcomes as a result of the care received.

Bias in pulse oximetry is the perfect case study of a well-trusted technology leading to significant patient harm. During the COVID-19 pandemic, many clinicians and patients were using oximeter technology for the first time and were not trained to spot factors, like melanin in the skin, that cause inaccurate measurements and impact patient care. Issues were largely not attributed to the device. This then leads to underreporting of adverse events to the FDA — which is already a problem due to the voluntary nature of adverse-event reporting. 

Even when problems are ultimately identified, the federal government is slow to respond. The pulse oximeter’s limitations in monitoring oxygenation levels across diverse skin tones was identified as early as the 1990s. 34 years later, despite repeated follow-up studies indicating biases, no manufacturer has incorporated skin-tone-adjusted calibration algorithms into pulse oximeters. It required the large Sjoding study, and the media coverage it garnered around delayed care and unnecessary deaths, for the FDA to issue a safety communication and begin reviewing the regulation.

Other areas of HHS are stepping up to address issues of bias in deployed technologies. A new ruling by the HHS Office of Civil Rights (OCR) on Section 1557 of the Affordable Care Act requires covered providers and institutions (i.e. any receiving federal funding) to identify their use of patient care decision support tools that directly measure race, color, national origin, sex, age, or disability, and to make reasonable efforts to mitigate the risk of discrimination from their use of these tools. Implementation of this rule will depend on OCR’s enforcement, and yet it provides another route to address bias in algorithmic tools.

Differential access to medical innovation is a form of bias

Americans face wildly different levels of access to new medical innovations. As many new innovations have high cost points, these drugs, devices, and algorithms exist outside the price range of many patients, smaller healthcare institutions and federally funded healthcare service providers, including the Veterans Health Administration, federally qualified health centers and the Indian Health Service. Emerging care-delivery strategies might not be covered by Medicare and Medicaid, meaning that patients insured by CMS cannot access the most cutting-edge treatments. Finally, the shift to digital health, spurred by COVID-19, has compromised access to healthcare in rural communities without reliable broadband access. 

Finally, the Advanced Research Projects Agency for Health (ARPA-H) has a commitment to have all programs and projects consider equity in their design. To fulfill ARPA-H’s commitment, there is a need for action to ensure that medical technologies are developed fairly, tested with rigor, deployed safely, and made affordable and accessible to everyone.

Plan of Action

The next Administration should launch “Healthcare Innovation for All Americans” (HIAA), a whole of government initiative to improve health outcomes by ensuring Americans have access to bias-free medical technologies. Through a comprehensive approach that addresses bias in all medical technology sectors, at all stages of the commercialization pipeline, and in all geographies, the initiative will strive to ensure the medical-innovation ecosystem works for all. HIAA should be a joint mandate of Health and Human Services (HHS) and the Office of Science Technology and Policy (OSTP) to work with federal agencies on priorities of equity, non-discrimination per Section 1557 of the Affordable Care Act and increasing access to medical innovation, and initiative leadership should sit at both HHS and OSTP. 

This initiative will require involvement of multiple federal agencies, as summarized in the table below. Additional detail is provided in the subsequent sections describing how the federal government can mitigate bias in the development phase; testing, regulation, and approval phases; and market deployment and evaluation phases.

Three guiding principles should underlie the initiative:

1. Equity and non-discrimination should drive action. Actions should seek to improve the health of those who have been historically excluded from medical research and development. We should design standards that repair past exclusion and prevent future exclusion. 
2. Coordination and cooperation are necessary. The executive and legislative branches must collaborate to address the full scope of the problem of bias in medical technology, from federal processes to new regulations. Legislative leadership should task the Government Accountability Office (GAO) to engage in ongoing assessment of progress towards the goal of achieving bias-free and fair medical innovation.
3. Transparent, evidence-based decision making is paramount. There is abundant peer-reviewed literature that examines bias in drugs, devices, and algorithms used in healthcare settings — this literature should form the basis of a non-discrimination approach to medical innovation. Gaps in evidence should be focused on through deployed research funding. Moreover, as algorithms become ubiquitous in medicine, every effort should be made to ensure that these algorithms are trained on representative data of those experiencing a given healthcare condition.

Agency	Role
Advanced Research Projects Agency for Health (ARPA-H)	ARPA-H has committed to tackling health equity in biomedical research, and to aligning each project it undertakes with that goal. As such, ARPA-H should lead the charge in developing processes for equity in medical technology — from idea conceptualization to large-scale rollout — and serve as a model for other federally funded healthcare programs.
National Institute of Health (NIH)	NIH should fund research that addresses health-data gaps, investigates algorithmic and data bias, and assesses bias embedded in medical technical tools. Simultaneously, NIH should create standards for diversity in samples and/or datasets for preclinical research. Finally, NIH must strongly enforce the 1993 NIH Revitalization Act’s diversity provisions.
National Science Foundation (NSF)	NSF should collaborate with NIH on cross-agency programs that fund R&D specific to mitigating bias of technologies like AI.
Food and Drug Administration (FDA)	FDA should take a more active role in uncovering bias in medical innovation, given its role as a regulatory checkpoint for all new medical technologies. This should include more rigorous evaluation protocols as well as better tracking of emergent bias in medical technologies post-approval.
Assistant Secretary for Technology Policy (ASTP)	ASTP publishes standards for effective use of healthcare information technology that ensure quality care delivery. Their standards-setting should offer solutions for compliance with Section 1557 for novel AI/ML algorithms.
Centers for Medicare and Medicaid (CMS)	CMS oversees the coordination of coverage, coding, and payment processes with respect to new technologies and procedures. Thus, CMS should focus on ensuring all new technologies developed through federal funding, like those that will be built by ARPA-H and its industry partners, are covered by Medicare and Medicaid. In addition, CMS and its accrediting partners can require compliance with federal regulatory standards, which should be extended to assess medical technologies. Finally, CMS should assess how flawed medical technologies are being used to decide on medical care provision, and update guidelines.
Federal Trade Commission (FTC)	FTC should protect America’s medical technology consumers by auditing high-risk medical innovations, such as decision-making algorithms.
Agency for Healthcare Research and Quality (AHRQ)	The AHRQ, a component of HHS, should identify areas where technology bias is leading to disparate healthcare outcomes and report its findings to Congress, the White House, and agency leaders for immediate action.
Centers for Disease Control and Prevention (CDC)	CDC’s expertise in health-data collection should be mobilized to identify research and development gaps.
Department of Commerce (DOC)	Given its role in enforcing U.S. trade laws and regulations, DOC can do much to incentivize equity in medical device design and delivery. The National Institute of Standards and Technology (NIST) should play a key role in crafting standards for identifying and managing bias across key medical-technology sectors.
Department of Education (ED)	ED should work with medical schools to develop and implement learning standards and curricula on bias in medical technology.
Department of Defense (DOD)	DOD has formalized relationships with FDA to expedite medical products useful to American military personnel. As a DOD priority is to expand diversity and inclusion in the armed forces, these medical products should be assessed for bias that limits safety and efficacy.
Health Resources and Services Administration (HRSA)	HRSA should coordinate with federally qualified health centers on digital health technologies, taking advantage of the broadband expansion outlined in the Bipartisan Infrastructure Bill.
Veterans Affairs (VA) and the Veterans Health Administration (VHA)	The VA should work with ARPA-H and its industry partners to establish cost-effective rollout of new innovations to VA-run hospitals. VA should also use its procurement power to require diversity in the clinical trials of the drugs, devices, and algorithms it procures. VA could also use prize challenges to spur innovation.
Government Accountability Office (GAO)	The GAO should prepare a comprehensive roadmap for addressing bias endemic to the cycle of medical technology development, testing, and deployment, with a focus on mitigating bias in “black box” algorithms used in medical technology.
Office of Management and Budget (OMB)	OMB should work with HIAA leadership to design a budget for HIAA implementation, including for R&D funding, personnel for programmatic expansion, data collectives, education, and regulatory enforcement.
Office of Science Technology and Policy (OSTP)	OSTP should develop processes and standards for ensuring that individual rights are not violated by biased medical technologies. This work can build on the AI Bill of Rights Initiative.

Addressing bias at the development phase

The following actions should be taken to address bias in medical technology at the innovation phase:

• Enforce parity in government-funded research. For clinical research, NIH should examine the widespread lack of adherence to regulations requiring that government funded clinical trials report sex, racial or ethnicity, and age breakdown of trial participants. Funding should be reevaluated for non-compliant trials. For preclinical research, NIH should require gender parity in animal models and representation of diverse cell lines used in federally funded studies.

• Deploy funding to address research gaps. Where data sources for historically marginalized people are lacking, such as for women’s cardiovascular health, NIH should deploy strategic, targeted funding programs to fill these knowledge gaps. This could build on efforts like the Initiative on Women’s Health Research. Increased funding should include resources for underrepresented groups to participate in research and clinical trials through building capacity in community organizations. Results should be added to a publicly available database so they can be accessed by designers of new technologies. Funding programs should also be created to fill gaps in technology, such as in diagnostics and treatments for high-prevalence and high-burden uterine diseases like endometriosis (found in 10% of reproductive-aged people with uteruses).      

• Invest in research into healthcare algorithms and databases. Given the explosion of algorithms in healthcare decision-making, NIH and NSF should launch a new research program focused on the study, evaluation, and application of algorithms in healthcare delivery, and on how artificial intelligence and machine learning (AI/ML) can exacerbate healthcare inequities. The initial request for proposals should focus on design strategies for medical algorithms that mitigate bias from data or model choices.

• Task ARPA-H with developing metrics for equitable medical technology development. ARPA-H should prioritize developing a set of procedures and metrics for equitable development of medical technology. Once developed, these processes should be rapidly deployed across ARPA-H, as well as published for potential adoption by additional federal agencies, industry, and other stakeholders. ARPA-H could also collaborate with NIST on standards setting with NIST and ASTP on relevant standards setting. For instance, NIST has developed an AI Risk Management Framework and the ONC engages in setting standards that achieve equity by design. CMS could use resultant standards for Medicare and Medicaid reimbursements.

• Leverage procurement as a demand-signal for medical technologies that work for diverse populations. As the nation’s largest healthcare system, the Veterans Health Administration (VHA) can generate demand-signals for bias-free medical technologies through its procurement processes and market-shaping mechanisms. For example, the VA could put out a call for a pulse oximeter that works equally well across the entire range of human skin pigmentation and offer contracts for the winning technology.

Addressing bias at the testing, regulation, and approval phases

The following actions should be taken to address bias in medical innovation at the testing, regulation, and approval phases:

• Raise the threshold for FDA evaluation of devices and algorithms. Equivalency necessary to receive 510(k) clearance should be narrowed. For algorithms, this would involve consideration of whether the datasets for machine learning tactics used by the new device and its predicate are similar. For devices (including those that use algorithms), this would require tightening the definition of “same intended use” (currently defined as a technology having the same functionality as one previously approved by the FDA) as well as eliminating the approval of new devices with “different technological characteristics” (the application of one technology to a new area of treatment in which that technology is untested). 

• Evaluate FDA’s guidance on specific technology groups for equity. Requirements for the safety of a given drug, medical device, or algorithm should have the statistical power necessary to detect differences between demographic groups and represent all end-users of the technology..

• Establish a data bank for auditing medical algorithms. The newly established Office of Digital Transformation within the FDA should create a “data bank” of healthcare images and datasets representative of the U.S. population, which could be done in partnership with the All of Us program. Medical technology developers could use the data bank to assess the performance of medical algorithms across patient populations. Regulators could use the data bank to ground claims made by those submitting a technology for FDA approval.

• Allow data submitted to the FDA to be examined by the broader scientific community. Currently, data submitted to the FDA as part of its regulatory-approval process is kept as a trade secret and not released pre-authorization to researchers. Releasing the data via an FDA-invited “peer review” step in the regulation of high-risk technologies, like automated decision-making algorithms, Class III medical devices, and drugs, will ensure that additional, external rigor is applied to the technologies that could cause the most harm due to potential biases.

• Establish an enforceable AI Bill of Rights. The federal government and Congress should create protections for necessary uses of artificial intelligence (AI) identified by OSTP. Federally funded healthcare centers, like facilities part of the Veterans Health Administration, could refuse to buy software or technology products that violate this “AI Bill of Rights” through changes to federal acquisition regulation (FAR).

Addressing bias at the market deployment and evaluation phases 

• Strengthen reporting mechanisms at the FDA. Healthcare providers, who are often closest to the deployment of medical technologies, should be made mandatory reporters to the FDA of all witnessed adverse events related to bias in medical technology. In addition, the FDA should require the inclusion of unique device identifiers (UDIs) in adverse-response reporting. Using this data, Congress should create a national and publicly accessible registry that uses UDIs to track post-market medical outcomes and safety. 

• Require impact assessments of deployed technologies. Congress must establish systems of accountability for medical technologies, like algorithms, that can evolve over time. Such work could be done by passing the Algorithmic Accountability Act which would require companies that create “high-risk automated decision systems” to conduct impact assessments reviewed by the FTC as frequently as necessary. 

• Assess disparities in patient outcomes to direct technical auditing. AHRQ should be given the funding needed to fully investigate patient-outcome disparities that could be caused by biases in medical technology, such as its investigation into the impacts of healthcare algorithms on racial and ethnic disparities. The results of this research should be used to identify technologies that the FDA should audit post-market for efficacy or the FTC should investigate. CMS and its accrediting agencies can monitor these technologies and assess whether they should receive Medicare and Medicaid funding. 

• Review reimbursement guidelines that are dependent on medical technologies with known bias. CMS should review its national coverage determinations for technologies, like pulse oximetry, that are known to perform differently across populations. For example, pulse oximeters can be used to determine home oxygen therapy provision, thus potentially excluding darkly-pigmented populations from receiving this benefit.

• Train physicians to identify bias in medical technologies and identify new areas of specialization. ED should work with medical schools to develop curricula training physicians to identify potential sources of bias in medical technologies and ensuring that physicians understand how to report adverse events to the FDA. In addition, ED should consider working with the American Medical Association to create new medical specialties that work at the intersection of technology and care delivery.

• Ensure that technologies developed by ARPA-H have an enforceable access plan. ARPA-H will produce cutting-edge technologies that must be made accessible to all Americans. ARPA-H should collaborate with the Center for Medicare and Medicaid Innovation to develop strategies for equitable delivery of these new technologies. A cost-effective deployment strategy must be identified to service federally-funded healthcare institutions like Veterans Health Administration hospitals and clinical, federally qualified health centers, and Indian Health Service.

• Create a fund to support digital health technology infrastructure in rural hospitals. To capitalize on the $65 billion expansion of broadband access allocated in the Bipartisan Infrastructure Bill, HRSA should deploy strategic funding to federally qualified health centers and rural health clinics to support digital health strategies — such as telehealth and mobile health monitoring —  and patient education for technology adoption.

A comprehensive road map is needed

The GAO should conduct a comprehensive investigation of “black box” medical technologies utilizing algorithms that are not transparent to end users, medical providers, and patients. The investigation should inform a national strategic plan for equity and non-discrimination in medical innovation that relies heavily on algorithmic decision-making. The plan should include identification of noteworthy medical technologies leading to differential healthcare outcomes, creation of enforceable regulatory standards, development of new sources of research funding to address knowledge gaps, development of enforcement mechanisms for bias reporting, and ongoing assessment of equity goals.

Timeline for action

Realizing HIAA will require mobilization of federal funding, introduction of regulation and legislation, and coordination of stakeholders from federal agencies, industry, healthcare providers, and researchers around a common goal of mitigating bias in medical technology. Such an initiative will be a multi-year undertaking and require funding to enact R&D expenditures, expand data capacity, assess enforcement impacts, create educational materials, and deploy personnel to staff all the above.

Near-term steps that can be taken to launch HIAA include issuing a public request for information, gathering stakeholders, engaging the public and relevant communities in conversation, and preparing a report outlining the roadmap to accomplishing the policies outlined in this memo.

Conclusion

Medical innovation is central to the delivery of high-quality healthcare in the United States. Ensuring equitable healthcare for all Americans requires ensuring that medical innovation is equitable across all sectors, phases, and geographies. Through a bold and comprehensive initiative, the next Administration can ensure that our nation continues leading the world in medical innovation while crafting a future where healthcare delivery works for all.

This action-ready policy memo is part of Day One 2025 — our effort to bring forward bold policy ideas, grounded in science and evidence, that can tackle the country’s biggest challenges and bring us closer to the prosperous, equitable and safe future that we all hope for whoever takes office in 2025 and beyond.

Frequently Asked Questions

How will the success of HIAA be evaluated?

HIAA will be successful when medical policies, projects, and technologies yield equitable health care access, treatment, and outcomes. For instance, success would yield the following outcomes:

• Representation in preclinical and clinical research equivalent to the incidence of a studied condition in the general population.


• Research on a disease condition funded equally per affected patient.


• Existence of data for all populations facing a given disease condition.


• Medical algorithms that have equal efficacy across subgroup populations.


• Technologies that work equally well in testing as they do when deployed to the market.


• Healthcare technologies made available and affordable to all care facilities.

read more

Why does this memo propose an expansive multi-agency effort instead of just targeting the FDA?

Regulation alone cannot close the disparity gap. There are notable gaps in preclinical and clinical research data for women, people of color, and other historically underrepresented groups that need to be filled. There are also historical biases encoded in AI/ML decision making algorithms that need to be studied and rectified. In addition, the FDA’s role is to serve as a safety check on new technologies — the agency has limited oversight over technologies once they are out on the market due to the voluntary nature of adverse reporting mechanisms. This means that agencies like the FTC and CMS need to be mobilized to audit high-risk technologies once they reach the market. Eliminating bias in medical technology is only possible through coordination and cooperation of federal agencies with each other as well as with partners in the medical device industry, the pharmaceutical industry, academic research, and medical care delivery.

read more

What challenges might the Administration encounter from industry in launching this initiative?

A significant focus of the medical device and pharmaceutical industries is reducing the time to market for new medical devices and drugs. Imposing additional requirements for subgroup analysis and equitable use as part of the approval process could work against this objective. On the other hand, ensuring equitable use during the development and approval stages of commercialization will ultimately be less costly than dealing with a future recall or a loss of Medicare or Medicaid eligibility if discriminatory outcomes are discovered.

read more

Is there bipartisan support to secure the funding for this initiative?

Healthcare disparities exist in every state in America and are costing billions a year in economic growth. Some of the most vulnerable people live in rural areas, where they are less likely to receive high-quality care because costs of new medical technologies are too high for the federally qualified health centers that serve one in five rural residents as well as rural hospitals. Furthermore, during continued use, a biased device creates adverse healthcare outcomes that cost taxpayers money. A technology functioning poorly due to bias can be expensive to replace. It is economically imperative to ensure technology works as expected, as it leads to more effective healthcare and thus healthier people.

read more

Policy entrepreneurs inside and outside of government, as well as other stakeholders and advocates, are often interested in expanding the use of effective methods across many or all federal agencies, because how the government accomplishes its mission is integral to what the government is able to produce in terms of outcomes for the public it serves. Adoption and use of promising new methods by federal agencies can be slowed by a number of factors that discourage risk-taking and experimentation, and instead encourage compliance and standardization, too often as a false proxy for accountability. As a result, many agency-specific and government-wide authorities for promising methods go under-considered and under-utilized. 

Policy entrepreneurs within center-of-government agencies (e.g., Executive Office of the President) are well-positioned to use a variety of policy levers and actions to encourage and accelerate federal agency adoption of promising and effective methods. Some interventions by center-of-government agencies are better suited to driving initial adoption, others to accelerating or maintaining momentum, and yet others to codifying and making adoption durable once widespread. Therefore, a policy entrepreneur interested in expanding adoption of a given method should first seek to understand the “adoption maturity” of that method and then undertake interventions appropriate for that stage of adoption. The arc of agency adoption of new methods can be long—measured in years and decades, not weeks and months. Policy entrepreneurs should be prepared to support adoption over similar timescales. In considering adoption maturity of a method of interest, policy entrepreneurs can also reference the ideas of Tom Kalil in a July 2024 Federation of American Scientists blog post on “Increasing the ‘Policy Readiness of Ideas,” which offers sample questions to ask about “the policy landscape surrounding a particular idea.”

As a case study for driving federal adoption of a new method, this paper looks back at actions that supported the widespread adoption of incentive prizes by most federal agencies over the course of fiscal years 2010 through 2020. Federal agency use of prizes increased from several incentive prize competitions being offered by a handful of agencies in the early 2000s to more than 2,000 prize competitions offered by over 100 federal agencies by the end of fiscal year 2022. These incentive prize competitions have helped federal agencies identify novel solutions and technologies, establish new industry benchmarks, pay only for results, and engage new talent and organizations. 

A summary framework below includes types of actions that can be taken by policy entrepreneurs within center-of-government agencies to support awareness, piloting, and ongoing use of new methods by federal agencies in the years ahead. (Federal agency program and project managers who seek to scale up innovative methods within their agencies are encouraged to reference related resources such as this article by Jenn Gustetic in the Winter 2018 Issues in Science and Technology: “Scaling Up Policy Innovations in the Federal Government: Lessons from the Trenches.”) 

Efforts to expand federal capacity through new and promising methods are worthwhile to ensure the federal government can use a full and robust toolbox of tactics to meet its varied goals and missions. 

OPPORTUNITIES AND CHALLENGES IN FEDERAL ADOPTION OF NEW METHODS

Opportunities for federal adoption and use of promising and effective methods

To address national priorities, solve tough challenges, or better meet federal missions to serve the public, a policy entrepreneur may aim to pilot, scale, and make lasting federal use of a specific method. 

A policy entrepreneur’s goals might include new ways for federal agencies to, for example:

• Catalyze the development, demonstration, and deployment of technology and novel solutions; 
• Acquire or commercialize products and services that meet government or national needs; 
• Engage and seek input from communities and the public;
• Deliver more effective, efficient, and equitable services and benefits; 
• Provide technical assistance to state, local, Tribal, and territorial governments;
• Retain and recruit talent for mission critical occupations or to fill federal skills gaps; 
• Assess and evaluate organizational health and performance or program-level outcomes; or
• Translate evidence to practice.

To support these and other goals, an array of promising methods exist and have been demonstrated, such as in other sectors like philanthropy, industry, and civil society, in state, local, Tribal, or territorial governments and communities, or in one or several federal agencies—with promise for beneficial impact if more federal agencies adopted these practices. Many methods are either specifically supported or generally allowable under existing government-wide or agency-specific authorities. 

Center-of-government agencies include components of the Executive Office of the President (EOP) like the Office of Management and Budget (OMB) and the Office of Science and Technology Policy (OSTP), as well as the Office of Personnel Management (OPM) and the General Services Administration (GSA). These agencies direct, guide, convene, support, and influence the implementation of law, regulation, and the President’s policies across all Federal agencies, especially the executive departments. An August 2016 report by the Partnership for Public Service and the IBM Center for the Business of Government noted that, “The Office of Management and Budget and other “center of government” agencies are often viewed as adding processes that inhibit positive change—however, they can also drive innovation forward across the government.”

A policy entrepreneur interested in expanding adoption of a given method through actions driven or coordinated by one or more center-of-government agencies should first seek to understand the “adoption maturity” of a given method of interest by assessing: (1) the extent that adoption of the method has already occurred across the federal interagency; (2) any real or perceived barriers to adoption and use; and (3) the robustness of existing policy frameworks and agency-specific and government-wide infrastructure and resources that support agency use of the method.

Challenges in federal adoption and use of new methods

Policy entrepreneurs are usually interested in expanding federal adoption of new methods for good reason: a focus on supporting and expanding beneficial outcomes. Effective leaders and managers across sectors understand the importance of matching appropriate and creative tactics with well-defined problems and opportunities. Ideally, leaders are picking which tactic or tool to use based on their expert understanding of the target problem or opportunity, not using a method solely because it is novel or because it is the way work has always been done in the past. Design of effective program strategies is supported by access to a robust and well-stocked toolbox of tactics. 

However, many currently authorized and allowable methods for achieving federal goals are generally underutilized in the implementation strategies and day-to-day tactics of federal agencies. Looking at the wide variety of existing authorities in law and the various flexibilities allowed for in regulation and guidance, one might expect agency tactics for common activities like acquisition or public comment to be varied, diverse, iterative, and even experimental in nature, where appropriate. In practice, however, agency methods are often remarkably homogeneous, repeated, and standardized.   

This underutilization of existing authorities and allowable flexibilities is due to factors such as:

• Comfort with existing methods among program and legal staff (“but that’s how we have always done it!”); 
• In turn, limited internal expertise on how to deploy specific new methods (“who in our agency knows how to do this well?”);
• Unclear legal authorities or lack of established agency policies or processes (“are we allowed to do that and, if so, where is that authority written?”)
• Unclear permission authorities and approval roles (“whose review and sign-off do I need to do that?”);
• Difficulties clearly defining the opportunity or problem to be addressed at an actionable level of specificity (“what does success look like?”);
• Concerns about perceived risks, such as the risk of funds going unawarded or effective solutions not being identified (“what if no one/nothing meets our target?”); 
• Oversight processes that seek out failure, flaws, and non-compliance and overreliance on strict procedures to ensure accountability (“who is responsible for this failure?” instead of “what can we learn from this for the future?”, and “have all the boxes been checked?” instead of “where might we start and how will we learn along the way?”); and
• Reluctance to define and implement meaningful performance indicators and assessment methods at the start of program design (“how will we know if this new method improves our outcomes compared to the status quo?”). 

Strategies for addressing challenges in federal adoption and use of new methods

Attention and action by center-of-government agencies often is needed to address the factors cited above that slow the adoption and use of new methods across federal agencies and to build momentum. The following strategies are further explored in the case study on federal use of incentive prizes that follows: 

• clarifying government-wide and agency-specific policies and processes;
• building awareness and fostering leadership and staff buy-in; 
• offering case studies, examples, and “how-to” playbooks; 
• creating connections among a federal community of practice 
• engaging external experts and practitioners; 
• removing identified barriers;
• increasing ambition through iterative experimentation; and 
• fostering an enterprise-wide learning culture that encourages experimentation, invests in evaluation, and manages risk.

Additional strategies can be deployed within federal agencies to address agency-level barriers and scale promising methods—see, for example, this article by Jenn Gustetic in the Winter 2018 Issues in Science and Technology: “Scaling Up Policy Innovations in the Federal Government: Lessons from the Trenches.” 

LOOKING BACK: A DECADE OF POLICY ACTIONS SUPPORTING EXPANDED FEDERAL USE OF INCENTIVE PRIZES

The use of incentive prizes is one method for open innovation that has been adopted broadly by most federal agencies, with extensive bipartisan support in Congress and with White House engagement across multiple administrations. In contrast to recognition prizes, such as the Nobel Prize or various presidential medals, which reward past accomplishments, incentive prizes specify a target, establish a judging process (ideally as objective as possible), and use a monetary prize purse and/or non-monetary incentives (such as media and online recognition, access to development and commercialization facilities, resources, or experts, or even qualification for certain regulatory flexibility) to induce new efforts by solvers competing for the prize. 

The use of incentive prizes by governments (and by high net worth individuals) to catalyze novel solutions certainly is not new. In 1795, Napoleon offered 12,000 francs to improve upon the prevailing food preservation methods of the time, with a goal of better feeding his army. Fifteen years later, confectioner Nicolas François Appert claimed the prize for his method involving heating, boiling and sealing food in airtight glass jars — the same basic technology still used to can foods. Dava Sobel’s book Longitude details how the rulers of Spain, the Netherlands, and Britain all offered separate prizes, starting in 1567, for methods of figuring out longitude at sea, and finally John Harrison was awarded Britain’s top longitude prize in 1773. In 1919, Raymond Orteig, a French-American hotelier, aviation enthusiast, and philanthropist, offered a $25,000 prize for the first person who could perform a nonstop flight between New York and Paris. The prize offer initially expired by 1924 without anyone claiming it. Given technological advances and a number of engaged pilots involved in trying to win the prize, Orteig extended the deadline by 5 years. By 1926, nine teams had come forward to formally compete, and the prize went to a little-known aviator named Charles Lindbergh, who attempted the flight in a custom-built plane known as the “Spirit of St. Louis.”

The U.S. Government did not begin to adopt the use of incentive prizes until the early 21st century, following a 1999 National Academy of Engineering workshop about the use of prizes as an innovation tool. In the first decade of the 2000s, the Defense Advanced Research Projects Agency (DARPA), the National Aeronautics and Space Administration (NASA), and the Department of Energy conducted a small number of pilot prize competitions. These early agency-led prizes focused on autonomous vehicles, space exploration, and energy efficiency, demonstrating a range of benefits to federal agency missions. 

Federal use of incentive prizes did not accelerate until, in the America COMPETES Reauthorization Act of 2010, Congress granted all federal agencies the authority to conduct prize competitions (15 USC § 3719). With that new authority in place, and with the support of a variety of other policy actions, federal use of incentive prizes reached scale, with over 2,000 prize competitions offered on Challenge.gov by over 100 federal agencies between the fiscal years 2010 and 2022. 

There certainly remains extensive opportunity to improve the design, rigor, ambition, and effectiveness of federal prize competitions. That said, there are informative lessons to be drawn from how incentive prizes evolved in the United States from a method used primarily outside of government, with limited pilots among a handful of early-adopter federal agencies, to a method being tried by many civil servants across an active interagency community of practice and lauded by administration leaders, bipartisan members of Congress, and external stakeholders alike. 

A summary follows of the strategies and tactics used by policy entrepreneurs within the EOP—with support and engagement from Congress as well as program managers and legal staff across federal agencies—that led to increased adoption and use of incentive prizes in the federal government.

Summary of strategies and policy levers supporting expanded use of incentive prizes

In considering how best to expand awareness, adoption, and use among federal agencies of promising methods, policy entrepreneurs might consider utilizing some or all of the strategies and policy levers described below in the incentive prizes example. Those strategies and levers are summarized generally in the table that follows. Some of the listed levers can advance multiple strategies and goals. This framework is intended to be flexible and to spark brainstorming among policy entrepreneurs, as they build momentum in the use of particular innovation methods. 

Policy entrepreneurs are advised to consider and monitor the maturity level of federal awareness, adoption, and use, and to adjust their strategies and tactics accordingly. They are encouraged to return to earlier strategies and policy levers as needed, should adoption and momentum lag, should agency ambition in design and implementation of initiatives be insufficient, or should concerns regarding risk management be raised by agencies, Congress, or stakeholders. 

Stage of Federal Adoption	Strategy	Types of Center-of-Government Policy Levers
Early – No or few Federal agencies using method	Understand federal opportunities to use method, and identify barriers and challenges	* Connect with early adopters across federal agencies to understand use of agency-specific authorities, identify pain points and lessons learned, and capture case studies (e.g., 2000-2009) * Engage stakeholder community of contractors, experts, researchers, and philanthropy * Look to and learn from use of method in other sectors (such as by philanthropy, industry, or academia) and document (or encourage third-party documentation of) that use and its known benefits and attributes (e.g., April 1999, July 2009) * Encourage research, analysis, reports, and evidence-building by National Academies, academia, think tanks, and other stakeholders (e.g., April 1999, July 2009, June 2014) * Discuss method with OMB Office of General Counsel and other relevant agency counsel * Discuss method with relevant Congressional authorizing committee staff * Host convenings that connect interested federal agency representatives with experts * Support and connect nascent federal “community of interest”
Early – No or few Federal agencies using method	Build interest among federal agencies	* Designate primary policy point of contact/dedicated staff member in the EOP (e.g., 2009-2017, 2017-2021) * Designate a primary implementation point of contact/dedicated staff at GSA and/or OPM * Identify leads in all or certain federal agencies * Connect topic to other administration policy agendas and strategies * Highlight early adopters within agencies in communications from center-of-government agencies to other federal agencies (and to external audiences) * Offer congressional briefings and foster bipartisan collaboration (e.g., 2015)
Early – No or few Federal agencies using method	Establish legal authorities and general administration policy	* Engage OMB Office of OMB General Counsel and OMB Legislative Review Division, as well as other relevant OMB offices and EOP policy councils * Identify existing general authorities and regulations that could support federal agency use of method (e.g., March 2010) * Establish general policy guidelines, including by leveraging Presidential authorities through executive orders or memoranda (e.g., January 2009) * Issue OMB directives on specific follow-on agency actions or guidance to support agency implementation (“M-Memos” or similar) (e.g., December 2009, March 2010, August 2011, March 2012) * Provide technical assistance to Congress regarding government-wide or agency-specific authority (or authorities) (e.g., June-July 2010, January 2011) * Delegate existing authorities within agencies (e.g., October 2011) * Encourage issuance of agency-specific guidance (e.g., October 2011, February 2014) * Include direction to agencies as part of broader Administration policy agendas (e.g., September 2009, 2011-2016)
Early – No or few Federal agencies using method	Remove barriers and “make it easier”	* Create a central government website with information for federal agency practitioners (such as toolkits, case studies, and trainings) and for the public (e.g., September 2010) * Create dedicated GSA schedule of vendors (e.g., July 2011) * Establish an interagency center of excellence (e.g., September 2011) * Encourage use of interagency agreements on design or implementation of pilot initiatives (e.g., September 2011) * Request agency budget submissions to OMB to support pilot use in President’s budget (e.g., December 2013)
Adoption well underway – Many federal agencies have begun to use method	Connect practitioners	* Launch a federal “community of practice” with support from GSA for meetings, listserv, and collaborative projects (e.g., April 2010, 2016, June 2019) * Host regular events, workshops, and conferences with federal agency and, where appropriate and allowable, seek philanthropic or nonprofit co-hosts (e.g., April 2010, June 2012, April 2015, March 2018, May 2022)
Adoption well underway – Many federal agencies have begun to use method	Strengthen agency infrastructure	* Foster leadership buy-in through briefings from White House/EOP to agency leadership, including members of the career senior executive service * Encourage agencies to dedicate agency staff and invest in prize design support within agencies * Encourage agencies to create contract vehicles as needed to support collaboration with vendors/ experts * Encourage agencies to develop intra-agency networks of practitioners and to provide external communications support and platforms for outreach * Request agency budget submissions to OMB for investments in agency infrastructure and expansion of use, to include in the President's budget where needed (e.g., 2012-2013), and request agencies otherwise accommodate lower-dollar support (such as allocation of FTEs) where possible within their budget toplines
Adoption well underway – Many federal agencies have begun to use method	Clarify existing policies and authorities	* Issue updated OMB, OSTP, or agency-specific policy guidance and memoranda as needed based on engagement with agencies and stakeholders (e.g.,: August 2011, March 2012) * Provide technical assistance to Congress on any needed updates to government-wide or agency-specific authorities (e.g., January 2017)
Adoption prevalent – Most if not all federal agencies have adopted, with a need to maintain use and momentum over time	Highlight progress and capture lessons learned	* Require regular reporting from agencies to EOP (OSTP, OMB, or similar) (e.g., April 2012, May 2022) * Require and take full advantage of regular reports to Congress (e.g., April 2012, December 2013, May 2014, May 2015, August 2016, June 2019, May 2022, April 2024) * Continue to capture and publish federal-use case studies in multiple formats online (e.g., June 2012) * Undertake research, evaluation, and evidence-building * Co-develop practitioner toolkit with federal agency experts (e.g., December 2016) * Continue to feature promising examples on White House/EOP blogs and communication channels (e.g., October 2015, August 2020) * Engage media and seek both general interest and targeted press coverage, including through external awards/honorifics (e.g., December 2013)
Adoption prevalent – Most if not all federal agencies have adopted, with a need to maintain use and momentum over time	Prepare for presidential transitions and document opportunities for future administrations	* Integrate go-forward proposals and lessons learned into presidential transition planning and transition briefings (e.g., June 2016-January 2017) * Brief external stakeholders and Congressional supporters on progress and future opportunities * Connect use of method to other, broader policy objectives and national priorities (e.g., August 2020, May 2022, April 2024)

Phases and timeline of policy actions advancing the adoption of incentive prizes by federal agencies

1. Growing number of incentive prizes offered outside government (early 2000s)

At the close of the 20th century, federal use of incentive prizes to induce activity toward targeted solutions was limited, though the federal government regularly utilized recognition prizes to reward past accomplishment. In October 2004, the $10 million Ansari XPRIZE—which was first announced in May 1996—was awarded by the XPRIZE Foundation for the successful flights of Spaceship One by Scaled Composites. Following the awarding of the Ansari XPRIZE and the extensive resulting news coverage, philanthropists and high net worth individuals began to offer prize purses to incentivize action on a wide variety of technology and social challenges. A variety of new online challenge platforms sprung up, and new vendors began offering consulting services for designing and hosting challenges, trends that lowered the cost of prize competition administration and broadened participation in prize competitions among thousands of diverse solvers around the world. This growth in the use of prizes by philanthropists and the private sector increased the interest of the federal government in trying out incentive prizes to help meet agency missions and solve national challenges. Actions during this period to support federal use of incentive prizes include:

• EXTERNAL REPORT/ANALYSIS (April 1999): In response to a request from the Clinton-Gore Administration’s National Economic Council, the National Academy of Engineering (NAE), with funding from the National Science Foundation, convened a workshop in April 1999 to “assess the potential value of federally sponsored prizes and contests in advancing science and technology in the public interest” and issued a brief summary report, which recommended “limited experiments” in the use of federally sponsored incentive prizes and encouraged both Congress and federal agencies “to take a flexible approach to the design and administration” of such prizes and that the use of incentive prizes be “evaluated at specified intervals by the agencies involved to determine their effectiveness and impact.”  

• AGENCY-SPECIFIC AUTHORITIES and PILOTS (early 2000s): During this period, though, only a few federal government agencies had (and still have) flexible agency-specific prize authorities that allowed them to pilot the use of incentive prizes to advance their missions, scan markets for new solutions, engage new solvers, and solve long-standing problems. Early examples include:

• Because of prizes authority provided to DARPA by Congress under 10 U.S.C. 2374a, enacted as part of the National Defense Authorization Act of 2000 in October 1999, DARPA was early in the federal prizes game, offering a series of challenges demonstrating the capabilities of autonomous vehicles in 2004 and 2005, called the DARPA Grand Challenges. 
• With authority from Congress, NASA began offering its ongoing series of Centennial Challenges starting in 2005, to directly engage the public in the process of advanced technology development related to problems of interest to NASA and the nation. 
• Congress also saw opportunity for the Department of Energy (DOE) to make progress on energy challenges through the use of incentive prizes, giving DOE authority through the Energy Independence and Security Act of 2007 to run a prize focused on efficient lighting call the “L-Prize” and also to run a series of “H-Prizes” to encourage research into the use of hydrogen as an energy carrier in a hydrogen economy.

2. Obama-Biden Administration Seeks to Expand Federal Prizes Through Administrative Action (2009-2010)

From the start of the Obama-Biden Administration, OSTP and OMB took a series of policy steps to expand the use of incentive prizes across federal agencies and build federal capacity to support those open-innovation efforts. Bipartisan support in Congress for these actions soon led to new legislation to further advance agency adoption of incentive prizes. Actions during this period to support federal use of incentive prizes include:

• PRESIDENTIAL DIRECTIVE (January 2009): On the first day of the Obama-Biden Administration on January 21, 2009, President Barack Obama signed a Memorandum on Transparency and Open Government, committing the Administration to creating a more transparent, participatory, and collaborative government. The memorandum directed that federal agencies “should offer Americans increased opportunities to participate in policymaking and to provide their Government with the benefits of their collective expertise and information.”

• EXTERNAL REPORT/ANALYSIS (July 2009): In July 2009, with funding from the Templeton Foundation, McKinsey issued a report called And the Winner Is… that documented the recent resurgence of incentive prizes and noted that over the past decade total prize purses across the large incentive prizes being offered had tripled to surpass $375 million. This report provided synthesis of learnings from recent prizes. For example, the report found that, “As Ken Davidian, formerly of the NASA Challenges, puts it, there are at least four core rewards that drive participants to compete for prizes: ‘goal, glory, guts, and gold—and gold is usually last.’ Or to be more precise (if less memorable), competitors are motivated by the intrinsic interest of a challenge, the recognition or prestige accompanying a winner, the challenge of the problem-solving process itself, and any material incentive. Which motives matter most, and in what mix, will vary depending on the problem—and the problem solver.”

• INCLUSION IN ADMINISTRATION POLICY AGENDA (September 2009): In addition, in September 2009, President Obama released his Strategy for American Innovation, developed by the National Economic Council (NEC) and OSTP. In that strategy, the President called for federal agencies to “take advantage of the expertise and insight of people both inside and outside the federal government, use high-risk, high-reward policy tools such as prizes and challenges to solve tough problems.” 

• OMB DIRECTIVE (December 2009): Responding to President Obama’s open government memorandum, on December 8, 2009, the OMB Director issued an Open Government Directive, which required executive departments and agencies to take specific actions to further the principles established by the open government memorandum. The directive charged the OMB Deputy Director for Management to “issue, through separate guidance or as part of any planned comprehensive management guidance, a framework for how agencies can use challenges, prizes, and other incentive-backed strategies to find innovative or cost-effective solutions to improving open government.” The directive also charged federal agencies to include in agency Open Government Plans “innovative methods, such as prizes and competitions, to obtain ideas from and to increase collaboration with those in the private sector, non-profit, and academic communities.”

• EOP LEADERSHIP ROLES (January 2009-January 2017): To support the development and implementation of these and other open-innovation policies, a member of the OSTP policy staff served as policy lead for open innovation, reporting to OSTP Deputy Director Tom Kalil, with additional leadership backing from OSTP Director John Holdren, the inaugural U.S. Chief Technology Officer (CTO) Aneesh Choprah, and then Deputy U.S. CTO Beth Noveck. During the Obama Administration, this OSTP open-innovation policy role was filled by Robynn Sturm Steffen from 2009-2011, the author (Cristin Dorgelo) from 2011-2014 until she became OSTP Chief of Staff, Jenn Gustetic on detail from NASA from 2014-2016, and Christofer Nelson from 2016 through the end of the Administration in January 2017. Sonal Shah as inaugural director of the White House Office of Social Innovation and Civic Participation in the Domestic Policy Council (DPC), and her successor Jonathan Greenblatt, also provided helpful leadership and led key stakeholder engagement efforts.

• OMB GUIDANCE (March 2010): Consistent with the Open Government Directive and the Strategy for American Innovation, OSTP worked closely with the OMB Office of the Deputy Director for Management and the OMB Office of General Counsel on developing guidance on incentive prizes for federal agencies. In March 2010, OMB issued OMB Memorandum M-10-11, Guidance on the Use of Challenges and Prizes to Promote Open Government. This memorandum included clarifications for federal agencies regarding what authorities they could use to offer prize purses, host and sponsor prize competitions, and engage third parties to operate such competitions. OMB Memorandum M-10-11 also established as Administration policy that agencies should:

• Utilize prizes and challenges as tools for advancing open government, innovation, and the agency’s mission;
• Identify and proactively address legal, regulatory, technical, and other barriers to the use of prizes and challenges;
• Select one or more individuals to identify and implement prizes and challenges, potentially in partnership with outside organizations, and to participate in a governmentwide “community of practice” led by OMB and OSTP; and
• Increase their capacity to support, design, and manage prizes, potentially in collaboration with external partners.

• CONVENING and COMMUNITY OF PRACTICE (April 2010): In April 2010, the White House (OSTP and the DPC Office of Social Innovation and Civic Participation) with the Case Foundation convened experts in incentive prize design and administration to share private-sector success stories with nearly 200 representatives from more than 35 federal agencies. This Summit on Promoting Innovation: Prizes, Challenges and Open Grantmaking served as a formal kickoff to a federal prizes and challenges community of practice, which GSA administered, and for which OSTP and OMB provided strategic direction, substantive agenda setting, and resourcing. With GSA’s support, this community of practice remains more than a decade later a valuable network for federal prize practitioners to connect and exchange promising practices and lessons learned. 

• TECHNICAL ASSISTANCE TO CONGRESS TO INFORM NEW LEGAL AUTHORITY (June-July 2010): Throughout this period, OSTP and OMB were collaborating with Congress to advance government-wide prize authority. On June 24, 2010, Senators Mark Pryor and Mark Warner introduced in the 111th Congress S.3530, the Reward Innovation in America Act of 2010. Drawing from this introduced bill, in July 2010, the Senate Commerce Committee approved the America COMPETES Reauthorization Act of 2010 with a provision providing government-wide prize authority. Specifically, P.L. 111-358 added Section 24 to the Stevenson-Wydler Technology Innovation Act of 1980 (15 USC § 3719). On July 27, 2010, the Director of OSTP thanked Senators Pryor and Warner for their leadership in a letter that OSTP also published on its blog, highlighting the steps the Administration was taking to set the stage for agencies to take full advantage of prizes authority should Congress move the authority forward.

• SHARED GOVERNMENT WEBSITE (September 2010 to Present): Responding to directives in M-10-11, with support from OSTP and OMB, and with leadership from internal champions, GSA in September 2010 launched a government-wide website called Challenge.gov to provide one place for citizen solvers to come and find the challenges being offered by federal agencies. Over time, Challenge.gov developed back-end capabilities to help federal program managers administer certain types of prize competition and expanded to serve as a knowledge repository for federal program managers looking for more information about designing and administering incentive prizes. Challenge.gov remains today the primary online hub for federally hosted prize competitions. 

3. Implementing New Government-Wide Prizes Authority Provided by the America COMPETES Act (2011-2016)

During this period of expansion in the federal use of incentive prizes supported by new government-wide prize authority provided by Congress, the Obama-Biden Administration continued to emphasize its commitment to the model, including as a key method for accomplishing administration priorities, including priorities related to open government and evidence-based decision making. Actions during this period to support federal use of incentive prizes include:

• NEW AUTHORITIES THROUGH LEGISLATION (January 2011): On January 4, 2011, President Obama signed into law the America COMPETES Reauthorization Act of 2010 (COMPETES Act), granting all agencies broad authority to conduct prize competitions to spur innovation, solve tough problems, and advance their core missions (Public Law 111-358). 

• GSA CONTRACT VEHICLE (July 2011 to Present): In July 2011, as called for by the new law, GSA established a contract vehicle—originally, Sub-Schedule 541 4G, now maintained as the Multiple Award Schedule, 541613, Professional Services – Marketing and Public Relations—to help federal agencies access private-sector technical assistance and consulting support for incentive prizes. Because prize competitions were still an emerging practice both inside and outside of the federal government, this contract vehicle allowed federal agencies to access experts and consultants who were building capacity for prizes across sectors and identifying what works in prize design and operations.

• OMB GUIDANCE (August 2011): In August 2011, OMB’s General Counsel and Chief Information Officer issued a memorandum on the new COMPETES prize authority to agency general counsels and CIOs, which OMB developed hand-in-hand with OSTP. The memorandum included a concise summary of the COMPETES Act’s new prizes authorities and requirements, and it provided guidance to agencies in their implementation of the prize authority found in this legislation. It also addressed an array of frequently asked questions raised by agencies, including agency questions about the new authority to conduct prizes up to $50 million with existing appropriations, as well as the new authorities to: accept private-sector funds for the design, administration, or prize purse of a competition; to partner with nonprofits and tap the expertise of for-profits for successful implementation; and to co-sponsor with another agency.

• AGENCY-SPECIFIC POLICIES and DELEGATION OF AUTHORITY (October 2011): Following issuance of this government-wide guidance, with the support of OSTP and OMB, agencies began to establish strategies and policies to further accelerate widespread use of the new prize authority granted to them under COMPETES. For example, the Department of Health and Human Services (HHS) was at the forefront of agency implementation efforts. On October 12, 2011, HHS Secretary Kathleen Sebelius issued a memorandum notifying the Department of the new prize authority provided under the America COMPETES Reauthorization Act, outlining the Department’s strategy to optimize the use of prize competitions, and calling on the heads of HHS operating and staff divisions to forecast their future use of prize competitions to stimulate innovation in advancing the agency’s mission. Secretary Sebelious also issued a formal delegation of the new prize authority in the Federal Register.

• CENTER OF EXCELLENCE and INTERAGENCY AGREEMENTS (November 2011): In November 2011, OSTP worked with NASA to launch a Center of Excellence for Collaborative Innovation (CoECI), which was co-founded by Jason Crusan and Jeff Davis. As NASA continued to mature the use of challenges and crowdsourcing methods as a new tool in its toolkit, OSTP encouraged NASA to assist other federal agencies in the use of crowdsourced challenges to solve tough, mission-critical problems. This included the new Center working with federal agencies on challenge design through interagency agreements, and supporting those agencies with accessing NASA’s contracts with prize platforms, including for algorithm and apps challenges and for ideation challenges. 

• OMB GUIDANCE (March 2012): Throughout 2011, OSTP met regularly with agencies, holding regular phone calls and in-person meetings—often at agency offices—with agency leadership, counsel, and program managers who were considering the use of prizes or undertaking prize design. In these interactions, OSTP listened to agencies, asked clarifying questions, and tracked common issues and challenges. For example, OSTP heard in these agency conversations a variety of agency questions regarding the Paperwork Reduction Act and its intersection with prize authorities. OSTP then collaborated with OMB to assess these issues and determine how clarifying guidance could support agency implementation of prizes. On March 1, 2012, OMB Office of Information and Regulatory Affairs (OIRA) issued a Frequently Asked Questions summary to address common agency questions. 

• AGENCY REPORTING TO EOP and REPORT TO CONGRESS (April 2012): In April 2012, the Obama-Biden Administration released a first report to Congress on federal use of incentive prizes in fiscal year 2011, as required by the America COMPETES Act. These reports to Congress and related reporting from federal agencies to OSTP—initially on an annual cycle and now biennial—have been an essential mechanism for tracking federal agency prize activity and capturing case studies and outcomes. The work led by OSTP with GSA to standardize how federal agencies tracked indicators and metrics regarding federal incentive prizes supported both progress tracking over time as well as storytelling by the Administration and prize supporters in Congress. The reports have also recorded steps taken by OSTP, GSA, and other agencies like NASA to build government-wide capacity and infrastructure related to prizes, and the steps taken within agencies to establish policies and processes to ease the use of prizes and remove barriers.

• CONVENING and ONLINE CASE STUDIES AND RESOURCES (June 2012): In June 2012, OSTP, the Case Foundation, and the Joyce Foundation hosted a day-long conference called Collaborative Innovation: Public Sector Prizes, which brought together hundreds of public- and private-sector practitioners to share case studies, research, and lessons learned. The event was partially live streamed, and resulted in a large amount of case studies and video resources that were available on the Case Foundation website, now archived.

• INCLUSION IN ADMINISTRATION POLICY AGENDA (2011-2016): The use of incentive prizes was included and encouraged throughout a variety of Administration policy agendas, year over year. For example, commitments related to prizes and challenges were included in each of the United States Open Government biennial National Action Plans issued during the Obama-Biden Administration, to maintain momentum and highlight this body of work to the international open government community. 

• GUIDANCE ON AGENCY BUDGET PROPOSALS (2012-2013): OSTP also collaborated with OMB to ensure that agencies considered and submitted to OMB budget proposals to expand their use of incentive prizes. On May 18, 2012, OMB issued Memorandum M-12-14 on the Use of Evidence and Evaluation in the 2014 Budget, which directed, “Agencies should also consider using the new authority under the America COMPETES legislation to support incentive prizes of up to $50 million. Like Pay for Success, well designed prizes and challenges can yield a very high return on the taxpayer dollar.” For agencies seeking to learn more about prizes, the memo noted, “The Office of Science and Technology Policy has created a ‘community of practice’ for agency personnel involved in designing and managing incentive prizes.” On July 26, 2013, OMB issued Memorandum M-13-17 on Next Steps in the Evidence and Innovation Agenda, OMB encouraged federal agencies to develop budget proposals “that focus Federal dollars on effective practices while also encouraging innovation in service delivery” and specifically mentioned incentive prizes as an encouraged pay-for-performance strategy. 

• REPORT TO CONGRESS (December 2013): On December 27, 2013, OSTP released a second report to Congress of federal use of incentive prizes under the COMPETES Act authority, and other authorities, in fiscal year 2012.

• LIFTING UP EFFORT FOR EXTERNAL RECOGNITION (December 2013): On January 24, 2014, Harvard University’s Ash Center for Democratic Governance and Innovation announced Challenge.gov as winner of the prestigious 2013 “Innovations in American Government Award” in honor of exemplary service and creativity in the public interest. The Obama Administration through GSA nominated Challenge.gov for this external honor to raise awareness and increase attention for federal prizes. 

• AGENCY-SPECIFIC POLICIES (February 2014): Agencies continued to state and clarify their internal policies and processes related to incentive prizes. For example, on February 12, 2014, the NASA Administrator issued an agency-wide policy directive—still in place today and last updated in June 2023—to encourage “the use of challenges, prize competitions, and crowdsourcing activities at all levels of the Agency to further its mission.”

• REPORT TO CONGRESS (May 2014): On May 7, 2014, OSTP released a third report to Congress on federal use of incentive prizes, focused on fiscal year 2013. This report found an 85 percent annual increase in prizes run under all legal authorities, an over 50 percent increase in the number of prizes conducted under the authority provided by COMPETES increased by over 50 percent compared to fiscal year 2012 (and nearly six-fold compared to 2011), and an increase in the size of agency-sponsored prize purses has grown as well—11 prizes had prize purses of $100,000 or greater in fiscal year 2013. 

• EXTERNAL REPORT/ANALYSIS (June 2014): On June 19, 2014, Deloitte University Press released a report—informed by research involving prize practitioners across government—covering in depth the lessons learned and best practices identified from over 350 prizes conducted by the Federal government and over 50 prizes conducted by state, local, and philanthropic leaders. The report, titled The craft of Prize Design: lessons from the public sector, was produced by Doblin (Deloitte’s innovation practice), in collaboration with Bloomberg Philanthropies, the Case Foundation, the Joyce Foundation, the Knight Foundation, the Kresge Foundation, and the Rockefeller Foundation.

• REPORT TO CONGRESS (May 2015): On May 8, 2015, OSTP released a fourth report to Congress on federal use of incentive prizes, focused on fiscal year 2014. This report highlighted steps agencies were taking to support the use of incentive prizes across their components and divisions, from streamlining access to vendors to support the design and implementation of prize competitions through contract vehicles, creating internal working groups, designating points of contact, and creating internal and external communications tools. 

• CONVENING and WHITE HOUSE MICROSITE (October 2015): On Oct 7, 2015, five years after the launch of Challenge.gov, the White House, in conjunction with the Case Foundation, the Joyce Foundation, and Georgetown University hosted a conference called  “All Hands on Deck: Solving Complex Problems through Prizes and Challenges” to convene federal prize practitioners and catalyze the next generation of ambitious federal prizes. The following day, GSA brought together the federal community to recognize progress with an awards ceremony. By that point, more than 440 federal prizes had been offered, engaging more than 200,000 citizen solvers. Also in October 2015, OSTP and DPC collaborated on the launch of a WhiteHouse.gov microsite with information on federal use of incentive prizes.

• FOSTER BIPARTISAN CONGRESSIONAL SUPPORT (2015): During this period, bipartisan support for the use of incentive prizes by federal agencies continued. In 2015, a Congressional Prize Caucus with bipartisan sponsorship was held to increase awareness and encourage the use of prize competitions. Numerous pieces of legislation supporting prize competitions to fuel medical research were also passed (e.g., the 21st Century Cures Act [Public Law 114-255] included a provision on EUREKA Prize Competitions [42 U.S.C. 284et seq] that authorized the National Institutes of Health in the Department of Health and Human Services to conduct prize competitions to fuel medical research).

• REPORT TO CONGRESS and TRAINING (August 2016): In August 2016, OSTP released a fifth report to Congress on the federal use of prizes in fiscal year 2015. OSTP noted that Challenge.gov had, by August 2016, “featured more than 700 prize competitions and challenges—conducted under the authority provided by COMPETES and other authorities—from more than 100 Federal agencies, departments, and bureaus.” OSTP and GSA together had engaged more than 1,500 federal professionals in training on prize design and operations. 

• PRACTITIONER TOOLKIT (December 2016): In December 2016, building on the robust body of federal knowledge on prizes, OSTP and GSA with the federal Community of Practice on Prizes and Challenges issued a robust practitioner’s toolkit on Challenge.gov with a lot of how-to information and practical case studies. The toolkit was developed by an interagency team using insights drawn from experts across federal agencies. 
  
  

4. Maintaining Momentum in New Presidential Administrations

Support for federal use of incentive prizes continued beyond the Obama-Biden Administration foundational efforts. Leadership by federal agency prize leads was particularly important to support this momentum from administration to administration. Actions during the Trump-Pence and Biden-Harris Administrations to support federal use of incentive prizes include:

• INTEGRATION INTO PRESIDENTIAL TRANSITION PLANNING (June 2016 – January 2017): As the end of the Obama-Biden Administration neared, OSTP worked with GSA and federal agency prize leads to prepare for the upcoming presidential transition and ensure the agency leads felt prepared, empowered, and supported with agency-level policies and processes so they could continue to design and launch prize competitions as part of their ongoing regular course of business. OSTP also integrated incentive prizes into its transition communications as a recommendation for the Trump-Pence Administration to continue. For example, in its list of 100 examples of the Obama-Biden Administration putting science it its rightful place, issued in June 2016, OSTP included the following: 

Harnessed American ingenuity through increased use of incentive prizes. Since 2010, more than 80 Federal agencies have engaged 250,000 Americans through more than 700 challenges on Challenge.gov to address tough problems ranging from fighting Ebola, to decreasing the cost of solar energy, to blocking illegal robocalls. These competitions have made more than $220 million available to entrepreneurs and innovators and have led to the formation of over 275 startup companies with over $70 million in follow-on funding, creating over 1,000 new jobs.

In addition, in January 2017, the Obama-Biden Administration OSTP mentioned the use of incentive prizes in its public “exit memo” as a key “pay-for-performance” method in agency science and technology strategies that “can deliver better results at lower cost for the American people,” and also noted:

Harnessing the ingenuity of citizen solvers and citizen scientists. The Obama Administration has harnessed American ingenuity, driven local innovation, and engaged citizen solvers in communities across the Nation by increasing the use of open-innovation approaches including crowdsourcing, citizen science, and incentive prizes. Following guidance and legislation in 2010, over 700 incentive prize competitions have been featured on Challenge.gov from over 100 Federal agencies, with steady growth every year.

• TECHNICAL ASSISTANCE TO CONGRESS ON UPDATES TO AUTHORITIES (January 2017): On January 6, 2017, the American Innovation and Competitiveness Act (AICA) was signed into law by President Obama [Public Law 114-329]. Reflecting extensive, multi-year staff-level engagement among Congressional staff and experts at the Obama-Biden Administration’s OSTP and OMB, the AICA updated the government-wide authority that previously had been granted to federal agencies by the COMPETES Act. The aim of the updates were to encourage more ambitious interagency and cross-sector partnerships (and co-funding, with explicit authority to solicit funds for federal prizes from beyond the federal government) in the design and administration of prize competitions, and to eliminate unnecessary administrative burden, among other changes. 

• EOP LEADERSHIP ROLES (2017-2021): During the Trump-Pence Administration, support for federal agency use of prizes and challenges continued. In the EOP, Matt Lira, then Special Assistant to the President for Innovation Policy in the White House Office of American Innovation, Michael Kratsios, then Deputy Assistant to the President and Deputy U.S. Chief Technology Officer, and others in OSTP engaged with agencies to maintain momentum and identify new opportunities for the effective application of the COMPETES Act prize authority. 

• CONVENING (March 2018): In March 2018, the White House hosted a “Fostering Innovation with Prizes and Challenges” roundtable with then Secretary of Energy Rick Perry and other leaders. The White House confirmed to participants that, “the Trump Administration strongly supports efforts by Federal agencies to host prizes and challenges, particularly those that leverage COMPETES Act authority, to address some of the Nation’s most pressing issues.”

• REPORT TO CONGRESS AND COMMUNITY OF PRACTICE (June 2019): During the Trump-Pence Administration, the federal prizes and challenges community of practice supported by GSA continued a network and active email list exchange of more than 730 current and prospective challenge managers in the Federal space. These agency prize practitioners were and continue to be essential to forward progress in the use of incentive prizes, in the federal government and beyond. OSTP’s fiscal year 2017-2018 biennial report to Congress on agency use of prizes, issued in June 2019 and the sixth such report, noted that, “monitoring the proliferation of State and local crowdsourcing initiatives, Challenge.gov expanded the email list to State and local government prize practitioners in 2018, inviting exchange and opening avenues for partnership.”

• CONNECTIONS TO NATIONAL PRIORITIES AND LEVERAGING WHITE HOUSE COMMUNICATION CHANNELS (August 2020): As the nation faced the COVID-19 pandemic, and as federal agencies responded to emerging challenges and sought to meet urgent needs during the ongoing public health crisis, they turned to incentive prizes as one tool for connecting with solvers across the country and identifying promising solutions. On August 12, 2020, then Director of OSTP Kelvin Droegemeier issued a memorandum to federal agencies highlighting nine prize competitions launched by agencies related to COVID-19 and calling on agencies to “double-down” on their deployment of prizes to meet the challenges of COVID-19. The memo also noted that OSTP was convening open innovation working groups to support these efforts and planning to host a series of webinars with Challenge.gov. The White House also issued a Fact Sheet communicating agency prize competition and open innovation activities related to COVID-19, with incentive prizes being used to catalyze advances in testing technologies, computational models, mental health services, ventilators, and needs of frontline health care workers.

• CONVENING (May 2022): OSTP and GSA collaborated on hosting an Open Innovation Forum to bring together practitioners of incentive prizes, citizen science, and crowdsourcing from across government and other sectors.

• AGENCY REPORTING TO EOP, REPORT TO CONGRESS, AND CONNECTIONS TO POLICY AGENDAS (May 2022 and April 2024): The Biden-Harris Administration has supported the continued use of incentive prizes by federal agencies as part of its commitment to expanding and improving public engagement in the work of the federal government. 

• On May 4, 2022, OSTP released its seventh report to Congress on federal incentive prize competitions (and the second that also included a focus on citizen science and crowdsourcing activities alongside incentive prizes). In releasing the report, OSTP noted in a blog post, “This new report details recent Federal efforts to stimulate innovation and partnership and expand the American public’s participation in science. These developments are aligned with the Biden-Harris Administration’s commitment to advancing equity in the science and technology ecosystem, including OSTP’s Time is Now Initiative, and recently released Equity Action Plan.” This report also reflected a new and more robust survey approach used by OSTP and GSA to collect information from agencies about federal incentive prizes, as well as continued efforts among federal agencies to streamline the use of incentive prizes and reduce or remove barriers. 
• On April 16, 2024, OSTP released its eighth report to Congress on Federal incentive prize competitions (and the third that also included a focus on citizen science and crowdsourcing). The report connected the continued growth in the use of incentive prizes by federal agencies to a broader Administration-wide “movement towards improving and expanding participation and engagement in not only government research and development, but also government processes more broadly.”

By the end of fiscal year 2022, federal agencies had hosted over 2,000 prize competitions on Challenge.gov, since its launch in 2010. OSTP, GSA, and NASA CoECI had provided training to well over 2,000 federal practitioners during that same period. 

Number of Federal Prize Competitions by Authority FY14-FY22

Source: Office of Science and Technology Policy. Biennial Report on “IMPLEMENTATION OF FEDERAL PRIZE AND CITIZEN SCIENCE AUTHORITY: FISCAL YEARS 2021-22.” April 2024.

Federal Agency Practices to Support the Use of Prize Competitions

Source: Office of Science and Technology Policy. Biennial Report on “IMPLEMENTATION OF FEDERAL PRIZE AND CITIZEN SCIENCE AUTHORITY: FISCAL YEARS 2019-20.” March 2022. 

CONCLUSION

Over the span of a decade, incentive prizes had moved from a tool used primarily outside of the federal government to one used commonly across federal agencies, due to a concerted, multi-pronged effort led by policy entrepreneurs and incentive prize practitioners in the EOP and across federal agencies, with bipartisan congressional support, crossing several presidential administrations. And yet, the work to support the use of prizes by federal agencies is not complete–there remains extensive opportunity to further improve the design, rigor, ambition, and effectiveness of federal prize competitions; to move beyond “ideas challenges” to increase the use of incentive prizes to demonstrate technologies and solutions in testbeds and real-world deployment scenarios; to train additional federal personnel on the use of incentive prizes; to learn from the results of federal incentive prizes competitions; and to apply this method to address pressing and emerging challenges facing the nation.

In applying these lessons to efforts to expand the use of other promising methods in federal agencies, policy entrepreneurs in center-of-government federal agencies should be strategic in the policy actions they take to encourage and scale method adoption, by first seeking to understand the adoption maturity of that method (as well as the relevant policy readiness) and then by undertaking interventions appropriate for that stage of adoption. With attention and action by policy entrepreneurs to address factors that discourage risk-taking, experimentation, and piloting of new methods by federal agencies, it will be possible for federal agencies to utilize a further-expanded strategic portfolio of methods to catalyze the development, demonstration, and deployment of technology and innovative solutions to meet agency missions, solve long-standing problems, and address grand challenges facing our nation. 

Photo by Nick Fewings

Despite significant advances in scientific tools and methods, the traditional, labor-intensive model of scientific research in materials discovery has seen little innovation. The reliance on highly skilled but underpaid graduate students as labor to run experiments hinders the labor productivity of our scientific ecosystem. An emerging technology platform known as Self-Driving Labs (SDLs), which use commoditized robotics and artificial intelligence for automated experimentation, presents a potential solution to these challenges.

SDLs are not just theoretical constructs but have already been implemented at small scales in a few labs. An ARPA-E-funded Grand Challenge could drive funding, innovation, and development of SDLs, accelerating their integration into the scientific process. A Focused Research Organization (FRO) can also help create more modular and open-source components for SDLs and can be funded by philanthropies or the Department of Energy’s (DOE) new foundation. With additional funding, DOE national labs can also establish user facilities for scientists across the country to gain more experience working with autonomous scientific discovery platforms. In an era of strategic competition, funding emerging technology platforms like SDLs is all the more important to help the United States maintain its lead in materials innovation.

Challenge and Opportunity

New scientific ideas are critical for technological progress. These ideas often form the seed insight to creating new technologies: lighter cars that are more energy efficient, stronger submarines to support national security, and more efficient clean energy like solar panels and offshore wind. While the past several centuries have seen incredible progress in scientific understanding, the fundamental labor structure of how we do science has not changed. Our microscopes have become far more sophisticated, yet the actual synthesizing and testing of new materials is still laboriously done in university laboratories by highly knowledgeable graduate students. The lack of innovation in how we historically use scientific labor pools may account for stagnation of research labor productivity, a primary cause of concerns about the slowing of scientific progress. Indeed, analysis of scientific literature suggests that scientific papers are becoming less disruptive over time and that new ideas are getting harder to find. The slowing rate of new scientific ideas, particularly in the discovery of new materials or advances in materials efficiency, poses a substantial risk, potentially costing billions of dollars in economic value and jeopardizing global competitiveness. However, incredible advances in artificial intelligence (AI) coupled with the rise of cheap but robust robot arms are leading to a promising new paradigm of material discovery and innovation: Self-Driving Labs. An SDL is a platform where material synthesis and characterization is done by robots, with AI models intelligently selecting new material designs to test based on previous experimental results. These platforms enable researchers to rapidly explore and optimize designs within otherwise unfeasibly large search spaces.

Today, most material science labs are organized around a faculty member or principal investigator (PI), who manages a team of graduate students. Each graduate student designs experiments and hypotheses in collaboration with a PI, and then executes the experiment, synthesizing the material and characterizing its property. Unfortunately, that last step is often laborious and the most time-consuming. This sequential method to material discovery, where highly knowledgeable graduate students spend large portions of their time doing manual wet lab work, rate limits the amount of experiments and potential discoveries by a given lab group. SDLs can significantly improve the labor productivity of our scientific enterprise, freeing highly skilled graduate students from menial experimental labor to craft new theories or distill novel insights from autonomously collected data. Additionally, they yield more reproducible outcomes as experiments are run by code-driven motors, rather than by humans who may forget to include certain experimental details or have natural variations between procedures.

Self-Driving Labs are not a pipe dream. The biotech industry has spent decades developing advanced high-throughput synthesis and automation. For instance, while in the 1970s statins (one of the most successful cholesterol-lowering drug families) were discovered in part by a researcher testing 3800 cultures manually over a year, today, companies like AstraZeneca invest millions of dollars in automation and high-throughput research equipment (see figure 1). While drug and material discovery share some characteristics (e.g., combinatorially large search spaces and high impact of discovery), materials R&D has historically seen fewer capital investments in automation, primarily because it sits further upstream from where private investments anticipate predictable returns. There are, however, a few notable examples of SDLs being developed today. For instance, researchers at Boston University used a robot arm to test 3D-printed designs for uniaxial compression energy adsorption, an important mechanical property for designing stronger structures in civil engineering and aerospace. A Bayesian optimizer was then used to iterate over 25,000 designs in a search space with trillions of possible candidates, which led to an optimized structure with the highest recorded mechanical energy adsorption to date. Researchers at North Carolina State University used a microfluidic platform to autonomously synthesize >100 quantum dots, discovering formulations that were better than the previous state of the art in that material family.

 AstraZeneca automation equipment

These first-of-a-kind SDLs have shown exciting initial results demonstrating their ability to discover new material designs in a haystack of thousands to trillions of possible designs, which would be too large for any human researcher to grasp. However, SDLs are still an emerging technology platform. In order to scale up and realize their full potential, the federal government will need to make significant and coordinated research investments to derisk this materials innovation platform and demonstrate the return on capital before the private sector is willing to invest it.

Other nations are beginning to recognize the importance of a structured approach to funding SDLs: University of Toronto’s Alan Aspuru-Guzik, a former Harvard professor who left the United States in 2018, has created an Acceleration Consortium to deploy these SDLs and recently received $200 million in research funding, Canada’s largest ever research grant. In an era of strategic competition and climate challenges, maintaining U.S. competitiveness in materials innovation is more important than ever. Building a strong research program to fund, build, and deploy SDLs in research labs should be a part of the U.S. innovation portfolio.

Plan of Action

While several labs in the United States are working on SDLs, they have all received small, ad-hoc grants that are not coordinated in any way. A federal government funding program dedicated to self-driving labs does not currently exist. As a result, the SDLs are constrained to low-hanging material systems (e.g., microfluidics), with the lack of patient capital hindering labs’ ability to scale these systems and realize their true potential. A coordinated U.S. research program for Self-Driving Labs should:

Initiate an ARPA-E SDL Grand Challenge: Drawing inspiration from DARPA’s previous grand challenges that have catalyzed advancements in self-driving vehicles, ARPA-E should establish a Grand Challenge to catalyze state-of-the-art advancements in SDLs for scientific research. This challenge would involve an open call for teams to submit proposals for SDL projects, with a transparent set of performance metrics and benchmarks. Successful applicants would then receive funding to develop SDLs that demonstrate breakthroughs in automated scientific research. A projected budget for this initiative is $30 million¹, divided among six selected teams, each receiving $5 million over a four-year period to build and validate their SDL concepts. While ARPA-E is best positioned in terms of authority and funding flexibility, other institutions like National Science Foundation (NSF) or DARPA itself could also fund similar programs.

Establish a Focused Research Organization to open-source SDL components: This FRO would be responsible for developing modular, open-source hardware and software specifically designed for SDL applications. Creating common standards for both the hardware and software needed for SDLs will make such technology more accessible and encourage wider adoption. The FRO would also conduct research on how automation via SDLs is likely to reshape labor roles within scientific research and provide best practices on how to incorporate SDLs into scientific workflows. A proposed operational timeframe for this organization is five years, with an estimated budget of $18 million over that time period. The organization would work on prototyping SDL-specific hardware solutions and make them available on an open-source basis to foster wider community participation and iterative improvement. A FRO could be spun out of the DOE’s new Foundation for Energy Security (FESI), which would continue to establish the DOE’s role as an innovative science funder and be an exciting opportunity for FESI to work with nontraditional technical organizations. Using FESI would not require any new authorities and could leverage philanthropic funding, rather than requiring congressional appropriations.

Provide dedicated funding for the DOE national labs to build self-driving lab user facilities, so the United States can build institutional expertise in SDL operations and allow other U.S. scientists to familiarize themselves with these platforms. This funding can be specifically set aside by the DOE Office of Science or through line-item appropriations from Congress. Existing prototype SDLs, like the Argonne National Lab Rapid Prototyping Lab or Berkeley Lab’s A-Lab, that have emerged in the past several years lack sustained DOE funding but could be scaled up and supported with only $50 million in total funding over the next five years. SDLs are also one of the primary applications identified by the national labs in the “AI for Science, Energy, and Security” report, demonstrating willingness to build out this infrastructure and underscoring the recognized strategic importance of SDLs by the scientific research community.

Frequently Asked Questions

What factors determine whether an SDL is appropriate for materials innovation?

As with any new laboratory technique, SDLs are not necessarily an appropriate tool for everything. Given that their main benefit lies in automation and the ability to rapidly iterate through designs experimentally, SDLs are likely best suited for:

• Material families with combinatorially large design spaces that lack clear design theories or numerical models (e.g., metal organic frameworks, perovskites)


• Experiments where synthesis and characterization are either relatively quick or cheap and are amenable to automated handling (e.g., UV-vis spectroscopy is relatively simple, in-situ characterization technique)


• Scientific fields where numerical models are not accurate enough to use for training surrogate models or where there is a lack of experimental data repositories (e.g., the challenges of using density functional theory in material science as a reliable surrogate model)

While these heuristics are suggested as guidelines, it will take a full-fledged program with actual results to determine what systems are most amenable to SDL disruption.

read more

What aren’t SDLs?

When it comes to exciting new technologies, there can be incentives to misuse terms. Self-Driving Labs can be precisely defined as the automation of both material synthesis and characterization that includes some degree of intelligent, automated decision-making in-the-loop. Based on this definition, here are common classes of experiments that are not SDLs:

• High-throughput synthesis, where synthesis automation allows for the rapid synthesis of many different material formulations in parallel (lacks characterization and AI-in-the-loop)


• Using AI as a surrogate trained over numerical models, which is based on software-only results. Using an AI surrogate model to make material predictions and then synthesizing an optimal material is also not a SDL, though certainly still quite the accomplishment for AI in science (lacks discovery of synthesis procedures and requires numerical models or prior existing data, neither of which are always readily available in the material sciences).

read more

Will SDLs “automate” away scientists? How will they change the labor structure of science?

SDLs, like every other technology that we have adopted over the years, eliminate routine tasks that scientists must currently spend their time on. They will allow scientists to spend more time understanding scientific data, validating theories, and developing models for further experiments. They can automate routine tasks but not the job of being a scientist.

However, because SDLs require more firmware and software, they may favor larger facilities that can maintain long-term technicians and engineers who maintain and customize SDL platforms for various applications. An FRO could help address this asymmetry by developing open-source and modular software that smaller labs can adopt more easily upfront.

read more

Jennifer Pahlka (@pahlkadot) is a FAS Senior Fellow and the author of Recoding America: Why Government is Failing in the Digital Age and How We Can Do Better. Here is Pahlka’s testimony about artificial intelligence presented today, January 10, 2024, to the full Senate Committee on Homeland Security and Government Affairs hearing on “Harnessing AI to Improve Government Services and Customer Experience”. More can be found here, here, and here.

---

How the U.S. government chooses to respond to the changes AI brings is indeed critical, especially in its use to improve government services and customer experience. If the change is going to be for the better (and we can’t afford otherwise) it will not be primarily because of how much or how little we constrain AI’s use. Constraints are an important conversation, and AI safety experts are better suited to discuss these than me. But we could constrain agencies significantly and still get exactly the bad outcomes that those arguing for risk mitigation want to avoid. We could instead direct agencies to dive headlong into AI solutions, and still fail to get the benefit that the optimists expect. The difference will come down to how much or how little capacity and competency we have to deploy these technologies thoughtfully.

There are really two ways to build capacity: having more of the right people doing the right things (including but not limited to leveraging technology like AI) and safely reducing the burdens we place on those people. AI, of course, could help reduce those burdens, but not without the workforce we need – one that understands the systems we have today, the policy goals we have set, and the technology we are bringing to bear to achieve those goals. Our biggest priority as a government should be building that capacity, working both sides of that equation (more people, less burden.)

Building that capacity will require bodies like the US Senate to use a wide range of the tools at its disposal to shape our future, and use them in a specific way. Those tools can be used to create mandates and controls on the institutions that deliver for the American people, adding more rules and processes for administrative agencies and others to comply with. Or they can be used to enable these institutions to develop the capacity they so desperately need and to use their judgment in the service of agreed-upon goals, often by asking what mandates and controls might be removed, rather than added. This critical AI moment calls for enablement.

The recent executive order on AI already provides some new controls and safeguards. The order strikes a reasonable balance between encouragement and caution, but I worry that some of its guidance will be applied inappropriately. For example, some government agencies have long been using AI for day to day functions like handwriting recognition on envelopes or improved search to retrieve evidence more easily, and agencies may now subject these benign, low-risk uses to red tape based on the order. Caution is merited in some places, and dangerous in others, where we risk moving backwards, not forward. What we need to navigate these frameworks of safeguard and control are people in agencies who can tell the difference, and who have the authority to act accordingly.

Moreover, in many areas of government service delivery, the status quo is frankly not worth protecting. We understandably want to make sure, for instance, that applicants for government benefits aren’t unfairly denied because of bias in algorithms. The reality is that, to take just one benefit, one in six determinations of eligibility for SNAP is substantively incorrect today. If you count procedural errors, the rate is 44%. Worse are the applications and adjudications that haven’t been decided at all, the ones sitting in backlogs, causing enormous distress to the public and wasting taxpayer dollars. Poor application of AI in these contexts could indeed make a bad situation worse, but for people who are fed up and just want someone to get back to them about their tax return, their unemployment insurance check, or even their company’s permit to build infrastructure, something has to change. We may be able to make progress by applying AI, but not if we double down on the remedies that failed in the Internet Age and hope they somehow work in the age of AI. We must finally commit to the hard work of building digital capacity.

Read the full testimony here.

Just like crop tops, flannel, and some truly unfortunate JNCO jeans that one of these authors wore in junior high, the trends of the 90’s are upon us again. In the innovation world, this means an outsized focus on tech-based economic development, the hottest new idea in economic development, circa 1995. This takes us back in time to fifteen years after the passage of the Bayh Dole Act, the federal legislation that granted ownership of federally funded research to universities. It was a time when the economy was expanding, dot-com growth was a boom, not a bubble, and we spent more time watching Saved by the Bell than thinking about economic impact. 

After the creation of tech transfer offices across the country and the benefit of time, universities were just starting to understand how much the changes wrought by Bayh-Dole would impact them (or not). A raft of optimistic investments in venture development organizations and state public-private partnerships swept the country, some of which (like Ben Franklin Technology Partners and BioSTL) are still with us today, and some of which (like the Kansas Technology Enterprise Center) have flamed out in spectacular fashion. All of a sudden, research seemed like a process to be harnessed for economic impact. Out of this era came the focus on “technology commercialization” that has captured the economic development imagination to this day. 

Commercialization, in the context of this piece, describes the process through which universities (or national labs) and the private sector collaborate to bring to the market technologies that were developed using federal funding. Unlike sponsored research and development, in which industry engages with universities from the beginning to fund and set a research agenda, commercialization brings in the private sector after the technology has been conceptualized. Successful commercialization efforts have now grown across the country, and we believe they can be described by four practical principles: 

• Principle 1: A strong research enterprise is a necessary precondition to building a strong commercialization pipeline.
• Principle 2: Commercialization via established businesses creates different economic impacts than commercialization via startups; each pathway requires fundamentally different support.
• Principle 3: Local context matters; what works in Boston won’t necessarily work in Birmingham. 
• Principle 4: Successful commercialization pipelines include interventions at the individual, institutional, and ecosystem level.

Principle 1: A strong research enterprise is a necessary precondition to building a strong commercialization pipeline.

The first condition necessary to developing a commercialization pipeline is a reasonably advanced research enterprise. While not every region in the U.S. has access to a top-tier research university, there are pockets of excellent research at most major U.S. R1 and R2 institutions. However, because there is natural attrition at each stage of the commercialization process (much like the startup process) a critical mass of novel, leading, and relevant research activity must exist in a given University. If that bar is assumed to be the ability to attract $10 million in research funding (the equivalent of winning 20-25 SBIR Phase 1 grants annually), that limits the number of schools that can run a fruitful commercialization pipeline to approximately 350 institutions, based on data from the NSF NCSES. A metro area should have at least one research institution that meets this bar in order to secure federal funding for the development of lab-to-market programs, though given the co-location of many universities, it is possible for some metro areas to have several such research institutions or none at all.

Principle 2: Commercialization via established businesses creates different economic impacts than commercialization via startups; each pathway requires fundamentally different support.

When talking about commercialization, it is also important to differentiate between whether a new technology is brought to market by a large, incumbent company or start-up. The first half of the commercialization process is the same for both: technology is transferred out of universities, national labs, and other research institutions through the process of registering, patenting, and licensing new intellectual property (IP). Once licensed, though, the commercialization pathway branches into two. 

With an incumbent company, whether or not it successfully brings new technology to the market is largely dependent on the company’s internal goals and willingness to commit resources to commercializing that IP. Often, incumbent companies will license patents as a defensive strategy in order to prevent competition with their existing product lines. As a result, license of a technology by an incumbent company cannot be assumed to represent a guarantee of commercial use or value creation.

The alternative pathway is for universities to license their IP to start-ups, which may be spun out of university labs. Though success is not guaranteed, licensing to these new companies is where new programs and better policies can actually make an impact. Start-ups are dependent upon successful commercialization and require a lot of support to do so. Policies and programs that help meet their core needs can play a significant role in whether or not a start-up succeeds. These core needs include independent space for demonstrating and scaling their product, capital for that work and commercialization activities (e.g. scouting customers and conducting sales), and support through mentorship programs, accelerators, and in-kind help navigating regulatory processes (especially in deep tech fields). 

Principle 3: Local context matters; what works in Boston won’t necessarily work in Birmingham. 

Unfortunately, many universities approach their tech transfer programs with the goal of licensing their technology to large companies almost exclusively. This arises because university technology transfer offices (TTOs) are often understaffed, and it is easier to license multiple technologies to the same large company under an established partnership than to scout new buyers and negotiate new contracts for each patent. The Bayh-Dole Act, which established the current tech transfer system, was never intended to subsidize the R&D expenditures of our nation’s largest and most profitable companies, nor was it intended to allow incumbents to weaponize IP to repel new market entrants. Yet, that is how it is being used today in practical application.

Universities are not necessarily to blame for the lack of resources, though. Universities spend on average 0.6% of their research expenditures on their tech transfer programs. However, there is a large difference in research expenditures between top universities that can attract over a billion in research funding and the average research university, and thus a large difference in the staffing and support of TTOs. State government funding for the majority of public research universities have been declining since 2008, though there has been a slight upswing since the pandemic, while R&D funding at top universities continues to increase. Only a small minority of TTOs bring in enough income from licensing in order to be self-sustaining, often from a single “blockbuster” patent, while the majority operate at a loss to the institution. 

To successfully develop innovation capacity in ecosystems around the country through increased commercialization activity, one must recognize that communities have dramatically different levels of resources dedicated to these activities, and thus, “best practices” developed at leading universities are seldom replicable in smaller markets. 

Principle 4: Successful commercialization pipelines include interventions at the individual, institutional, and ecosystem level.

As we’ve discussed at length in our FAS “systems-thinking” blog series, which includes a post on innovation ecosystems, a systems lens is fundamental to how we see the world. Thinking in terms of systems helps us understand the structural changes that are needed to change the conditions that we see playing out around us every day. When thinking about the structure of commercialization processes, we believe that intervention at various structural levels of a system is necessary to create progres on challenges that seem insurmountable at first—such as changing the cultural expectations of “success” that are so influential in the academic systems. Below we have identified some good practices and programs for supporting commercialization at the individual, institutional, and ecosystem level, with an emphasis on pathways to start-ups and entrepreneurship.

Practices and Programs Targeted at Individuals

University tech transfer programs are often reliant on individuals taking the initiative to register new IP with their TTOs. This requires individuals to be both interested enough in commercialization and knowledgeable enough about the commercialization potential of their research to pursue registration. Universities can encourage faculty to be proactive in pursuing commercialization through recognizing entrepreneurial activities in their hiring, promotion and tenure guidelines and encouraging faculty to use their sabbaticals to pursue entrepreneurial activities. An analog to the latter at national laboratories are Entrepreneurial Leave Programs that allow staff scientists to take a leave of up to three years to start or join a start-up before returning to their position at the national lab.

Faculty and staff scientists are not the only source of IP though; graduate students and postdoctoral researchers produce much of the actual research behind new intellectual property. Whether or not these early-career researchers pursue commercialization activities is correlated with whether they have had research advisors who were engaged in commercialization. For this reason, in 2007, the National Research Foundation of Singapore established a joint research center with the Massachusetts Institute of Technology (MIT) such that by working with entrepreneurial MIT faculty members, researchers at major Singaporean universities would also develop a culture of entrepreneurship. Most universities likely can’t establish programs of this scale, but some type of mentorship program for early-career scientists pre-IP generation can help create a broader culture of translational research and technology transfer. Universities should also actively support graduate students and postdoctoral researchers in putting forward IP to their TTO. Some universities have even gone so far as to create funds to buy back the time of graduate students and postdocs from their labs and direct that time to entrepreneurial activities, such as participating in an iCorps program or conducting primary market research.  

Some universities have even gone so far as to create funds to buy back the time of graduate students and postdocs from their labs and direct that time to entrepreneurial activities, such as participating in an iCorps program or conducting primary market research.

Once IP has been generated and licensed, many universities offer mentorship programs for new entrepreneurs, such as MIT’s Venture Mentorship Services. Outside of universities, incubators and accelerators provide mentorship along with funding and/or co-working spaces for start-ups to grow their operation. Hardware-focused start-ups especially benefit from having a local incubator or accelerator, since hard-tech start-ups attract significantly less venture capital funding and support than digital technology start-ups, but require larger capital expenditures as they scale. Shared research facilities and testbeds are also crucial for providing hard-tech start-ups with the lab space and equipment to refine and scale their technologies.

For internationally-born entrepreneurs, an additional consideration is visa sponsorship. International graduate students and postdocs that launch start-ups need visa sponsors in order to stay in the United States as they transition out of academia. Universities that participate in the Global Entrepreneur in Residence program help provide H-1B visas for international entrepreneurs to work on their start-ups in affiliation with universities. The university benefits in return by attracting start-ups to their local community that then generate economic opportunities and help create an entrepreneurial ecosystem.

Practices and Programs Targeted at Institutions

As mentioned in the beginning, one of the biggest challenges for university tech transfer programs is understaffed TTOs and small patent budgets. On average, TTOs have only four people on staff, who can each file a handful of patents a year, and budgets for the legal fees on even fewer patents. Fully staffing TTOs can help universities ensure that new IP doesn’t slip through the cracks due to a lack of capacity for patenting or licensing activities. Developing standard term sheets for licensing agreements can also reduce administrative burden and make it easier for TTOs to establish new partnerships.

Instead of TTOs, some universities have established affiliated technology intermediaries, which are organizations that take on the business aspects of technology commercialization. For example, the Wisconsin Alumni Research Foundation (WARF) was launched as an independent, nonprofit corporation to manage the University of Wisconsin–Madison’s vitamin D patents and invest the resulting revenue into future research at the university. Since its inception 90 years ago, WARF has provided $2.3 billion in grants to the university and helped establish 60 start-up companies. 

In general, universities need to be more consistent about collecting and reporting key performance indicators for TTOs outside of the AUTM framework, such as the number of unlicensed patents and the number of products brought to the market using licensed technologies. In particular, universities should disaggregate metrics for licensing and partnerships between companies less than five years old and those greater than five years old so that stakeholders can see whether there is a difference in commercialization outcomes between incumbent and start-up licensees.

Practices and Programs Targeted at Innovation Ecosystems

Innovation ecosystems are made up of researchers, entrepreneurs, corporations, the workforce, government, and sources of capital. Geographic proximity through co-locating universities, corporations, start-ups, government research facilities, and other stakeholder institutions can help foster both formal and informal collaboration and result in significant technology-driven economic growth and benefits. Co-location may arise organically over time or result from the intentional development of research parks, such as the NASA Research Park. When done properly, the work of each stakeholder should advance a shared vision. This can create a virtuous cycle that attracts additional talent and stakeholders to the shared vision and can integrate with more traditional attraction and retention efforts. One such example is the co-location of the National Bio- and Agro-Defense Facility in Manhattan, KS, near the campus of Kansas State University. After securing that national lab, the university made investments in additional BSL-2, 3 and 3+ research facilities including the Biosecurity Research Institute and its Business Development Module. The construction and maintenance of those facilities required the creation of new workforce development programs to train HVAC technicians that manage the independent air handling capabilities of the labs and train biomanufacturing workers, which was then one of the selling points for the successful campaign for the relocation of corporation Scorpius Biologics to the region. At best, all elements of an innovation ecosystem are fueled by a research focus and the commercialization activity that it provides. 

For regions that find themselves short of the talent they need, soft-landing initiatives can help attract domestic and international entrepreneurs, start-ups, and early-stage firms to establish part of their business in a new region or to relocate entirely. This process can be daunting for early-stage companies, so soft-landing initiatives aim to provide the support and resources that will help an early-stage company acclimatize and thrive in a new place. These initiatives help to expand the reach of a community, create a talent base, and foster the conditions for future economic growth and benefits.

Alongside the creation of innovation ecosystems should be the establishment of “scale-up ecosystems” focused on developing and scaling new manufacturing processes necessary to mass produce the new technologies being developed. This is often an overlooked aspect of technology development in the United States, and supply chain shocks over the past few years have shone a light on the need to develop more local manufacturing supply chains. Fostering the growth of manufacturing alongside technology innovation can (1) reduce the time cycling between product and process development in the commercialization process, (2) capture the “learning by doing” benefits from scaling the production of new technologies, and (3) replenish the number of middle-income jobs that have been outsourced over the past few decades. 

Any way you slice it, commercialization capacity is one clear and critical input to a successful innovation ecosystem. However, it’s not the only element that’s important. A strong startup commercialization effort, standing alone, without the corporate, workforce, or government support that it needs to build a vibrant ecosystem around its entrepreneurs, might wane with time or simply be very successful at shipping spinouts off to a coastal hotspot. Building a commercialization pipeline is not, nor has it ever been, a one-size-fits-all solution for ecosystem building. 

It may even be something we’ve over-indexed on, given the widespread adoption of tech-based economic development strategies. One significant reason for this is the fact that entrepreneurship via commercialization is most open to those who already have access to a great deal of privilege–who have attained, or are on the path to, graduate degrees in STEM fields critical to our national competitiveness. If you’ve already earned a Ph.D. in machine learning, chances are your future is looking pretty bright—with or without entrepreneurial opportunity involved. To truly reap the economic benefits of commercialization activity (and the startups it creates), we need to aggressively implement programs, training, and models that change the demographics of who gets to commercialize technology, not just how they do it. To shape this, we’ll need to change the conditions for success for early-career researchers and reconsider the established model of how we mentor and train the next generation of scientists and engineers–you’ll hear more from us on these topics in future posts!

This article was written as a part of the FRO Forecasting project, a partnership between the Federation of American Scientists and Metaculus. This project aims to conduct a pilot study of forecasting as an approach for assessing the scientific and societal value of proposals for Focused Research Organizations. To learn more about the project, see the press release here. To participate in the pilot, you can access the public forecasting tournament here.

The United States federal government is the single largest funder of scientific research in the world. Thus, the way that science agencies like the National Science Foundation and the National Institutes of Health distribute research funding has a significant impact on the trajectory of science as a whole. Peer review is considered the gold standard for evaluating the merit of scientific research proposals, and agencies rely on peer review committees to help determine which proposals to fund. However, peer review has its own challenges. It is a difficult task to balance science agencies’ dual mission of protecting government funding from being spent on overly risky investments while also being ambitious in funding proposals that will push the frontiers of science, and research suggests that peer review may be designed more for the former rather than the latter. We at FAS are exploring innovative approaches to peer review to help tackle this challenge.

Biases in Peer Review

A frequently echoed concern across the scientific and metascientific community is that funding agencies’ current approach to peer review of science proposals tends to be overly risk-averse, leading to bias against proposals that entail high risk or high uncertainty about the outcomes. Reasons for this conservativeness include reviewer preferences for feasibility over potential impact, contagious negativity, and problems with the way that peer review scores are averaged together.

This concern, alongside studies suggesting that scientific progress is slowing down, has led to a renewed effort to experiment with new ways of conducting peer review, such as golden tickets and lottery mechanisms. While golden tickets and lottery mechanisms aim to complement traditional peer review with alternate means of making funding decisions — namely individual discretion and randomness, respectively — they don’t fundamentally change the way that peer review itself is conducted. 

Traditional peer review asks reviewers to assess research proposals based on a rubric of several criteria, which typically include potential value, novelty, feasibility, expertise, and resources. These criteria are given a score based on a numerical scale; for example, the National Institutes of Health uses a scale from 1 (best) to 9 (worst). Reviewers then provide an overall score that need not be calculated in any specific way based on the criteria scores. Next, all of the reviewers convene to discuss the proposal and submit their final overall scores, which may be different from what they submitted prior to the discussion. The final overall scores are averaged across all of the reviewers for a specific proposal. Proposals are then ranked based on their average overall score and funding is prioritized for those ranked before a certain cutoff score, though depending on the agency, some discretion by program administrators is permitted.  

The way that this process is designed allows for the biases mentioned at the beginning—reviewer preferences for feasibility, contagious negativity, and averaging problems—to influence funding decisions. First, reviewer discretion in deciding overall scores allows them to weigh feasibility more heavily than potential impact and novelty in their final scores. Second, when evaluations are discussed reviewers tend to adjust their scores to better align with their peers. This adjustment tends to be greater when correcting in the negative direction than in the positive direction, resulting in a stronger negative bias. Lastly, since funding tends to be quite limited, cutoff scores tend to be quite close to the best score. This means that even if almost all of the reviewers rate a proposal positively, one very negative review can potentially bring the average below the cutoff.

Designing a New Approach to Peer Review

In 2021, the researchers Chiara Franzoni and Paula Stephan published a working paper arguing that risk in science results from three sources of uncertainty: uncertainty of research outcomes, uncertainty of the probability of success, and uncertainty of the value of the research outcomes. To comprehensively and consistently account for these sources of uncertainty, they proposed a new expected utility approach to peer review evaluations, in which reviewers are asked to

1. Identify the primary expected outcome of a research proposal and, optionally, a potential secondary outcome;
2. Assess the probability between 0 to 1 of achieving each expected outcome (P(j); and
3. Assess the value of achieving each expected outcome (uj) on a numerical scale (e.g., 0 to 100).

From this, the total expected utility can be calculated for each proposal and used to rank them.¹ This systematic approach addresses the first bias we discussed by limiting the extent to which reviewers’ preferences for more feasible proposals would impact the final score of each proposal.

We at FAS see a lot of potential in Franzoni and Stephan’s expected value approach to peer review, and it inspired us to design a pilot study using a similar approach that aims to chip away at the other biases in review.

To explore potential solutions for negativity bias, we are taking a cue from forecasting by complementing the peer review process with a resolution and scoring process. This means that at a set time in the future, reviewers’ assessments will be compared to a ground truth based on the actual events that have occurred (i.e., was the outcome actually achieved and, if so, what was its actual impact?). Our theory is that if implemented in peer review, resolution and scoring could incentivize reviewers to make better, more accurate predictions over time and provide empirical estimates of a committee’s tendency to provide overly negative (or positive) assessments, thus potentially countering the effects of contagion during review panels and helping more ambitious proposals secure support. 

Additionally, we sought to design a new numerical scale for assessing the value or impact of a research proposal, which we call an impact score. Typically, peer reviewers are free to interpret the numerical scale for each criteria as they wish; Franzoni and Stephan’s design also did not specify how the numerical scale for the value of the research outcome should work. We decided to use a scale ranging from 1 (low) to 10 (high) that was base 2 exponential, meaning that a proposal that receives a score of 5 has double the impact of a proposal that receives a score of 4, and quadruple the impact of a proposal that receives a score of 3.

Figure 1. Plot demonstrating the exponential nature of the impact score.

Table 1. Example of how to interpret the impact score.

Score	Impact
1	None or negative
2	Minimal
3	Low or mixed
4	Moderate
5	High
6	Very high
7	Exceptional
8	Transformative
9	Revolutionary
10	Paradigm-shifting

The choice of an exponential scale reflects the tendency in science for a small number of research projects to have an outsized impact (Figure 2), and provides more room at the top end of the scale for reviewers to increase the rating of the proposals that they believe will have an exceptional impact. We believe that this could help address the last bias we discussed, which is that currently, bad scores are more likely to pull a proposal’s average below the cutoff than good scores are likely to pull a proposal’s average above the cutoff.

Figure 2. Citation distribution of accepted and rejected journal articles

Source: Siler, Lee, and Bero (2014)

We are now piloting this approach on a series of proposals in the life sciences that we have collected for Focused Research Organizations, a new type of non-profit research organization designed to tackle challenges that neither academia or industry is incentivized to work on. The pilot study was developed in collaboration with Metaculus, a forecasting platform and aggregator, and will be hosted on their website. We welcome subject matter experts in the life sciences — or anyone interested! — to participate in making forecasts on these proposals here. Stay tuned for the results of this pilot, which we will publish in a report early next year.

Summary

Public and active transportation are not equally accessible to all Americans. Due to a lack of sufficient infrastructure and reliable service for public transportation and active modes like biking, walking, and rolling, Americans must often depend on personal vehicles for travel to work, school, and other activities. During the past two years, Congress has allocated billions of dollars to equitable infrastructure, public transportation upgrades, and decreasing greenhouse gas pollution from transportation across the United States. The Department of Transportation (DOT) and its agencies should embrace innovation and partnerships to continue to increase active and public transportation across the country. The DOT should require grant applications for funding to discuss cross-agency collaborations, partner with the Department of Housing and Urban Development (HUD) to organize prize competitions, encourage public-private partnerships (P3s), and work with the Environmental Protection Agency (EPA) to grant money for transit programs through the Greenhouse Gas Reduction Fund. 

Challenge and Opportunity

Historically, U.S. investment in transportation has focused on expanding and developing highways for personal vehicle travel. As a result, 45% of Americans do not have access to reliable and safe public transportation, perpetuating the need for single-use vehicles for almost half of the country. The EPA reports that transportation accounts for 29% of total U.S. greenhouse gas emissions, with 58% of those emissions coming from light-duty cars. This large share of nationwide emissions from personal vehicles has short- and long-term climate impacts. 

Investments in green public and active transit should be a priority for the DOT in transitioning away from a personal-vehicle-dominated society and meeting the Biden Administration’s “goals of a 100% clean electrical grid by 2035 and net-zero carbon emissions by 2050.” Public and active transportation infrastructure includes bus systems, light rail, bus rapid transit, bike lanes, and safe sidewalks. Investments in public and active transportation should go towards a combination of electrifying existing public transportation, such as buses; improving and expanding public transit to be more reliable and accessible for more users; constructing bike lanes; developing community-owned bike share programs; and creating safe walking corridors. 

In addition to reducing carbon emissions, improved public transportation that disincentivizes personal vehicle use has a variety of co-benefits. Prioritizing public and active transportation could limit congestion on roads and lower pollution. Fewer vehicles on the road result in less tailpipe emissions, which “can trigger health problems such as aggravated asthma, reduced lung capacity, and increased susceptibility to respiratory illnesses, including pneumonia and bronchitis.” This is especially important for the millions of people who live near freeways and heavily congested roads. 

Congestion can also be financially costly for American households; the INRIZ Global Traffic Scorecard reports that traffic congestion cost the United States $81 billion in 2022. Those costs include vehicle maintenance, fuel cost, and “lost time,” all of which can be reduced with reliable and accessible public and active transportation. Additionally, the American Public Transportation Association reports that every $1 invested in public transportation generates $5 in economic returns, measured by savings in time traveled, reduction in traffic congestion, and business productivity. Thus, by investing in public transportation, communities can see improvements in air quality, economy, and health.

Public transportation is primarily managed at the local and state level; currently, over  6000 local and state transportation agencies provide and oversee public transportation in their regions. Public transportation is funded through federal, state, and local sources, and transit agencies receive funding from “passenger fares and other operating receipts.” The Federal Transit Administration (FTA) distributes funding for transit through grants and loans and accounts for 15% of total income for transit agencies, including 31% of capital investments in transit infrastructure. Local and state entities often lack sufficient resources to improve public transportation systems because of the uncertainty of ridership and funding streams.

Public-private partnerships can help alleviate some of these resource constraints because contracts can allow the private partner to operate public transportation systems. Regional and national collaboration across multiple agencies from the federal to the municipal level can also help alleviate resource barriers to public transit development. Local and state agencies do not have to work alone to improve public and active transportation systems. 

The following recommendations provide a pathway for transportation agencies at all levels of government to increase public and active transportation, resulting in social, economic, and environmental benefits for the communities they serve. 

Plan of Action

Recommendation 1. The FTA should require grant applicants for programs such as the Rebuilding American Infrastructure with Sustainability and Equity (RAISE) to define how they will work collaboratively with multiple federal agencies and conduct community engagement. 

Per the National Blueprint for Transportation Decarbonization, FTA staff should prioritize funding for grant applicants who successfully demonstrate partnerships and collaboration. This can be demonstrated, for example, with letters of support from community members and organizations for transit infrastructure projects. Collaboration can also be demonstrated by having applicants report clear goals, roles, and responsibilities for each agency involved in proposed projects. The FTA should: 

1. Develop a rubric for evaluating partnerships’ efficiency and alignment with national transit decarbonization goals. 
2. Create a tiered metrics system within the rubric that prioritizes grants for projects based on collaboration and reduction of greenhouse gas emissions in the transit sector.
3. Add a category to their Guidance Center on federal-state-local partnerships to provide insight on how they view successful collaboration. 

Recommendation 2. The DOT and HUD should collaborate on a prize competition to design active and/or public transportation projects to reduce traffic congestion. 

Housing and transportation costs are related and influence one another, which is why HUD is a natural partner. Funding can be sourced from the Highway Trust Fund, which the DOT has the authority to allocate up to “1% of the funds for research and development to carry out . . . prize competition program[s].”

This challenge should call on local agency partners to provide a design challenge or opportunity that impedes their ability to adopt transit-oriented infrastructure that could reduce traffic congestion. Three design challenges should be selected and publicly posted on the Challenge.gov website so that any individual or organization can participate. 

The goal of the prize competition is to identify challenges, collaborate, and share resources across agencies and communities to design transportation solutions. The competition would connect the DOT with local and regional planning and transportation agencies to solicit solutions from the public, whether from individuals, teams of individuals, or organizations. The DOT and HUD should work collaboratively to design the selection criteria for the challenge and select the winners. Each challenge winner would be provided with a financial prize of $250,000, and their idea would be housed on the DOT website as a case study that can be used for future planning decisions. The local agencies that provide the three design challenges would be welcome to implement the winning solutions.

Recommendation 3. Federal, state, and local government should increase opportunities for public-private partnerships (P3s). 

The financial investment required to develop active and public transportation infrastructure is a hurdle for many agencies. To address this issue, we make the following recommendations: 

• Currently, only 36 out of the 50 states have policies that allow the use of P3s. The remaining 14 states should pass legislation authorizing the use of P3s for public transportation projects so that they too can benefit from this financing model and access federal P3 funding opportunities. 
• In 2016, the DOT launched the Build America Bureau to assist with financing transportation projects. The Bureau administers the Transportation Infrastructure Finance and Innovation Act (TIFIA) program, which provides financial assistance through low-interest loans for infrastructure projects and leverages public-private partnerships to access additional private-sector funding. Currently, only about 30% of all loans through the TIFIA are used for public transit projects while 66% are used on tolls and highways. Local and regional agencies should use the TIFIA loan more to fund public and active transit projects.
• EPA should specify in its Greenhouse Gas Reduction Fund guidelines that public and active transit projects are eligible for investment from the fund and can leverage public and private partnerships. EPA is set to distribute $27 billion through the Fund for carbon pollution reduction: $20 billion will go towards nonprofit entities, such as green banks, that will leverage public and private investment to fund emissions reduction projects, with $8 billion allocated to projects in low-income and disadvantaged communities; $7 billion will go to state and local agencies and nonprofits in the form of grants or technical assistance to low-income and disadvantaged communities. EPA should encourage applicants to include public and active transportation projects, which can play a significant role in reducing carbon emissions and air pollution, in their portfolios. 

Conclusion

The road to decarbonizing the transportation sector requires public and active transportation. Federal agencies can allocate funding for public and active transit more effectively through the recommendations above. It’s time for the government to recognize public and active transportation as the key to equitable decarbonization of the transportation sector throughout the United States.

Frequently Asked Questions

What are examples of transit public-private partnerships (P3s)?

Most P3s in the United States are for highways, bridges, and roads, but there have been a few successful public transit P3s. In 2018 the City of Los Angeles joined LAX and LAX Integrated Express Solutions in a $4.9 billion P3 to develop a train system within the airport. This project aims to launch in 2024 to “enhance the traveler experience” and will “result in 117,000 fewer vehicle miles traveled per day” to the airport. This project is a prime example of how P3s can help reduce traffic congestion and enable and encourage the use of public transportation.

read more

How can P3s be further supported through federal policy beyond the recommendations in this memo?

In 2021, the Congressional Research Service released a report about public-private partnerships (3Ps) that highlights the role the federal government can play by making it easier for agencies to participate in P3s.

read more

What are examples of existing green banks and infrastructure banks?

The state of Michigan has a long history with its Michigan Saves program, the nation’s first nonprofit green bank, which provides funding for projects like rooftop solar or energy efficiency programs.

In California the California Alternative Energy and Advanced Transportation Financing Authority works “collaboratively with public and private partners to provide innovative and effective financing solutions” for renewable energy sources, energy efficiency, and advanced transportation and manufacturing technologies.

The Rhode Island Infrastructure Bank provides funding to municipalities, businesses, and homeowners for projects “including water and wastewater, roads and bridges, energy efficiency and renewable energy, and brownfield remediation.”

read more