Automating Scientific Discovery: A Research Agenda for Advancing Self-Driving Labs

Despite significant advances in scientific tools and methods, the traditional, labor-intensive model of scientific research in materials discovery has seen little innovation. The reliance on highly skilled but underpaid graduate students as labor to run experiments hinders the labor productivity of our scientific ecosystem. An emerging technology platform known as Self-Driving Labs (SDLs), which use commoditized robotics and artificial intelligence for automated experimentation, presents a potential solution to these challenges.

SDLs are not just theoretical constructs but have already been implemented at small scales in a few labs. An ARPA-E-funded Grand Challenge could drive funding, innovation, and development of SDLs, accelerating their integration into the scientific process. A Focused Research Organization (FRO) can also help create more modular and open-source components for SDLs and can be funded by philanthropies or the Department of Energy’s (DOE) new foundation. With additional funding, DOE national labs can also establish user facilities for scientists across the country to gain more experience working with autonomous scientific discovery platforms. In an era of strategic competition, funding emerging technology platforms like SDLs is all the more important to help the United States maintain its lead in materials innovation.

Challenge and Opportunity

New scientific ideas are critical for technological progress. These ideas often form the seed insight to creating new technologies: lighter cars that are more energy efficient, stronger submarines to support national security, and more efficient clean energy like solar panels and offshore wind. While the past several centuries have seen incredible progress in scientific understanding, the fundamental labor structure of how we do science has not changed. Our microscopes have become far more sophisticated, yet the actual synthesizing and testing of new materials is still laboriously done in university laboratories by highly knowledgeable graduate students. The lack of innovation in how we historically use scientific labor pools may account for stagnation of research labor productivity, a primary cause of concerns about the slowing of scientific progress. Indeed, analysis of scientific literature suggests that scientific papers are becoming less disruptive over time and that new ideas are getting harder to find. The slowing rate of new scientific ideas, particularly in the discovery of new materials or advances in materials efficiency, poses a substantial risk, potentially costing billions of dollars in economic value and jeopardizing global competitiveness. However, incredible advances in artificial intelligence (AI) coupled with the rise of cheap but robust robot arms are leading to a promising new paradigm of material discovery and innovation: Self-Driving Labs. An SDL is a platform where material synthesis and characterization is done by robots, with AI models intelligently selecting new material designs to test based on previous experimental results. These platforms enable researchers to rapidly explore and optimize designs within otherwise unfeasibly large search spaces.

Today, most material science labs are organized around a faculty member or principal investigator (PI), who manages a team of graduate students. Each graduate student designs experiments and hypotheses in collaboration with a PI, and then executes the experiment, synthesizing the material and characterizing its property. Unfortunately, that last step is often laborious and the most time-consuming. This sequential method to material discovery, where highly knowledgeable graduate students spend large portions of their time doing manual wet lab work, rate limits the amount of experiments and potential discoveries by a given lab group. SDLs can significantly improve the labor productivity of our scientific enterprise, freeing highly skilled graduate students from menial experimental labor to craft new theories or distill novel insights from autonomously collected data. Additionally, they yield more reproducible outcomes as experiments are run by code-driven motors, rather than by humans who may forget to include certain experimental details or have natural variations between procedures.

Self-Driving Labs are not a pipe dream. The biotech industry has spent decades developing advanced high-throughput synthesis and automation. For instance, while in the 1970s statins (one of the most successful cholesterol-lowering drug families) were discovered in part by a researcher testing 3800 cultures manually over a year, today, companies like AstraZeneca invest millions of dollars in automation and high-throughput research equipment (see figure 1). While drug and material discovery share some characteristics (e.g., combinatorially large search spaces and high impact of discovery), materials R&D has historically seen fewer capital investments in automation, primarily because it sits further upstream from where private investments anticipate predictable returns. There are, however, a few notable examples of SDLs being developed today. For instance, researchers at Boston University used a robot arm to test 3D-printed designs for uniaxial compression energy adsorption, an important mechanical property for designing stronger structures in civil engineering and aerospace. A Bayesian optimizer was then used to iterate over 25,000 designs in a search space with trillions of possible candidates, which led to an optimized structure with the highest recorded mechanical energy adsorption to date. Researchers at North Carolina State University used a microfluidic platform to autonomously synthesize >100 quantum dots, discovering formulations that were better than the previous state of the art in that material family.

These first-of-a-kind SDLs have shown exciting initial results demonstrating their ability to discover new material designs in a haystack of thousands to trillions of possible designs, which would be too large for any human researcher to grasp. However, SDLs are still an emerging technology platform. In order to scale up and realize their full potential, the federal government will need to make significant and coordinated research investments to derisk this materials innovation platform and demonstrate the return on capital before the private sector is willing to invest it.

Other nations are beginning to recognize the importance of a structured approach to funding SDLs: University of Toronto’s Alan Aspuru-Guzik, a former Harvard professor who left the United States in 2018, has created an Acceleration Consortium to deploy these SDLs and recently received $200 million in research funding, Canada’s largest ever research grant. In an era of strategic competition and climate challenges, maintaining U.S. competitiveness in materials innovation is more important than ever. Building a strong research program to fund, build, and deploy SDLs in research labs should be a part of the U.S. innovation portfolio.

Plan of Action

While several labs in the United States are working on SDLs, they have all received small, ad-hoc grants that are not coordinated in any way. A federal government funding program dedicated to self-driving labs does not currently exist. As a result, the SDLs are constrained to low-hanging material systems (e.g., microfluidics), with the lack of patient capital hindering labs’ ability to scale these systems and realize their true potential. A coordinated U.S. research program for Self-Driving Labs should:

Initiate an ARPA-E SDL Grand Challenge: Drawing inspiration from DARPA’s previous grand challenges that have catalyzed advancements in self-driving vehicles, ARPA-E should establish a Grand Challenge to catalyze state-of-the-art advancements in SDLs for scientific research. This challenge would involve an open call for teams to submit proposals for SDL projects, with a transparent set of performance metrics and benchmarks. Successful applicants would then receive funding to develop SDLs that demonstrate breakthroughs in automated scientific research. A projected budget for this initiative is $30 million1, divided among six selected teams, each receiving $5 million over a four-year period to build and validate their SDL concepts. While ARPA-E is best positioned in terms of authority and funding flexibility, other institutions like National Science Foundation (NSF) or DARPA itself could also fund similar programs.

Establish a Focused Research Organization to open-source SDL components: This FRO would be responsible for developing modular, open-source hardware and software specifically designed for SDL applications. Creating common standards for both the hardware and software needed for SDLs will make such technology more accessible and encourage wider adoption. The FRO would also conduct research on how automation via SDLs is likely to reshape labor roles within scientific research and provide best practices on how to incorporate SDLs into scientific workflows. A proposed operational timeframe for this organization is five years, with an estimated budget of $18 million over that time period. The organization would work on prototyping SDL-specific hardware solutions and make them available on an open-source basis to foster wider community participation and iterative improvement. A FRO could be spun out of the DOE’s new Foundation for Energy Security (FESI), which would continue to establish the DOE’s role as an innovative science funder and be an exciting opportunity for FESI to work with nontraditional technical organizations. Using FESI would not require any new authorities and could leverage philanthropic funding, rather than requiring congressional appropriations.

Provide dedicated funding for the DOE national labs to build self-driving lab user facilities, so the United States can build institutional expertise in SDL operations and allow other U.S. scientists to familiarize themselves with these platforms. This funding can be specifically set aside by the DOE Office of Science or through line-item appropriations from Congress. Existing prototype SDLs, like the Argonne National Lab Rapid Prototyping Lab or Berkeley Lab’s A-Lab, that have emerged in the past several years lack sustained DOE funding but could be scaled up and supported with only $50 million in total funding over the next five years. SDLs are also one of the primary applications identified by the national labs in the “AI for Science, Energy, and Security” report, demonstrating willingness to build out this infrastructure and underscoring the recognized strategic importance of SDLs by the scientific research community.

Frequently Asked Questions
What factors determine whether an SDL is appropriate for materials innovation?

As with any new laboratory technique, SDLs are not necessarily an appropriate tool for everything. Given that their main benefit lies in automation and the ability to rapidly iterate through designs experimentally, SDLs are likely best suited for:



  • Material families with combinatorially large design spaces that lack clear design theories or numerical models (e.g., metal organic frameworks, perovskites)

  • Experiments where synthesis and characterization are either relatively quick or cheap and are amenable to automated handling (e.g., UV-vis spectroscopy is relatively simple, in-situ characterization technique)

  • Scientific fields where numerical models are not accurate enough to use for training surrogate models or where there is a lack of experimental data repositories (e.g., the challenges of using density functional theory in material science as a reliable surrogate model)


While these heuristics are suggested as guidelines, it will take a full-fledged program with actual results to determine what systems are most amenable to SDL disruption.

What aren’t SDLs?

When it comes to exciting new technologies, there can be incentives to misuse terms. Self-Driving Labs can be precisely defined as the automation of both material synthesis and characterization that includes some degree of intelligent, automated decision-making in-the-loop. Based on this definition, here are common classes of experiments that are not SDLs:



  • High-throughput synthesis, where synthesis automation allows for the rapid synthesis of many different material formulations in parallel (lacks characterization and AI-in-the-loop)

  • Using AI as a surrogate trained over numerical models, which is based on software-only results. Using an AI surrogate model to make material predictions and then synthesizing an optimal material is also not a SDL, though certainly still quite the accomplishment for AI in science (lacks discovery of synthesis procedures and requires numerical models or prior existing data, neither of which are always readily available in the material sciences).

Will SDLs “automate” away scientists? How will they change the labor structure of science?

SDLs, like every other technology that we have adopted over the years, eliminate routine tasks that scientists must currently spend their time on. They will allow scientists to spend more time understanding scientific data, validating theories, and developing models for further experiments. They can automate routine tasks but not the job of being a scientist.


However, because SDLs require more firmware and software, they may favor larger facilities that can maintain long-term technicians and engineers who maintain and customize SDL platforms for various applications. An FRO could help address this asymmetry by developing open-source and modular software that smaller labs can adopt more easily upfront.

FAS Senior Fellow Jen Pahlka testifies on Using AI to Improve Government Services

Jennifer Pahlka (@pahlkadot) is a FAS Senior Fellow and the author of Recoding America: Why Government is Failing in the Digital Age and How We Can Do Better. Here is Pahlka’s testimony about artificial intelligence presented today, January 10, 2024, to the full Senate Committee on Homeland Security and Government Affairs hearing on “Harnessing AI to Improve Government Services and Customer Experience”. More can be found here, here, and here.


How the U.S. government chooses to respond to the changes AI brings is indeed critical, especially in its use to improve government services and customer experience. If the change is going to be for the better (and we can’t afford otherwise) it will not be primarily because of how much or how little we constrain AI’s use. Constraints are an important conversation, and AI safety experts are better suited to discuss these than me. But we could constrain agencies significantly and still get exactly the bad outcomes that those arguing for risk mitigation want to avoid. We could instead direct agencies to dive headlong into AI solutions, and still fail to get the benefit that the optimists expect. The difference will come down to how much or how little capacity and competency we have to deploy these technologies thoughtfully.

There are really two ways to build capacity: having more of the right people doing the right things (including but not limited to leveraging technology like AI) and safely reducing the burdens we place on those people. AI, of course, could help reduce those burdens, but not without the workforce we need – one that understands the systems we have today, the policy goals we have set, and the technology we are bringing to bear to achieve those goals. Our biggest priority as a government should be building that capacity, working both sides of that equation (more people, less burden.)

Building that capacity will require bodies like the US Senate to use a wide range of the tools at its disposal to shape our future, and use them in a specific way. Those tools can be used to create mandates and controls on the institutions that deliver for the American people, adding more rules and processes for administrative agencies and others to comply with. Or they can be used to enable these institutions to develop the capacity they so desperately need and to use their judgment in the service of agreed-upon goals, often by asking what mandates and controls might be removed, rather than added. This critical AI moment calls for enablement.

The recent executive order on AI already provides some new controls and safeguards. The order strikes a reasonable balance between encouragement and caution, but I worry that some of its guidance will be applied inappropriately. For example, some government agencies have long been using AI for day to day functions like handwriting recognition on envelopes or improved search to retrieve evidence more easily, and agencies may now subject these benign, low-risk uses to red tape based on the order. Caution is merited in some places, and dangerous in others, where we risk moving backwards, not forward. What we need to navigate these frameworks of safeguard and control are people in agencies who can tell the difference, and who have the authority to act accordingly.

Moreover, in many areas of government service delivery, the status quo is frankly not worth protecting. We understandably want to make sure, for instance, that applicants for government benefits aren’t unfairly denied because of bias in algorithms. The reality is that, to take just one benefit, one in six determinations of eligibility for SNAP is substantively incorrect today. If you count procedural errors, the rate is 44%. Worse are the applications and adjudications that haven’t been decided at all, the ones sitting in backlogs, causing enormous distress to the public and wasting taxpayer dollars. Poor application of AI in these contexts could indeed make a bad situation worse, but for people who are fed up and just want someone to get back to them about their tax return, their unemployment insurance check, or even their company’s permit to build infrastructure, something has to change. We may be able to make progress by applying AI, but not if we double down on the remedies that failed in the Internet Age and hope they somehow work in the age of AI. We must finally commit to the hard work of building digital capacity.

What Works in Boston, Won’t Necessarily Work in Birmingham: 4 Pragmatic Principles for Building Commercialization Capacity in Innovation Ecosystems

Just like crop tops, flannel, and some truly unfortunate JNCO jeans that one of these authors wore in junior high, the trends of the 90’s are upon us again. In the innovation world, this means an outsized focus on tech-based economic development, the hottest new idea in economic development, circa 1995. This takes us back in time to fifteen years after the passage of the Bayh Dole Act, the federal legislation that granted ownership of federally funded research to universities. It was a time when the economy was expanding, dot-com growth was a boom, not a bubble, and we spent more time watching Saved by the Bell than thinking about economic impact. 

After the creation of tech transfer offices across the country and the benefit of time, universities were just starting to understand how much the changes wrought by Bayh-Dole would impact them (or not). A raft of optimistic investments in venture development organizations and state public-private partnerships swept the country, some of which (like Ben Franklin Technology Partners and BioSTL) are still with us today, and some of which (like the Kansas Technology Enterprise Center) have flamed out in spectacular fashion. All of a sudden, research seemed like a process to be harnessed for economic impact. Out of this era came the focus on “technology commercialization” that has captured the economic development imagination to this day. 

Commercialization, in the context of this piece, describes the process through which universities (or national labs) and the private sector collaborate to bring to the market technologies that were developed using federal funding. Unlike sponsored research and development, in which industry engages with universities from the beginning to fund and set a research agenda, commercialization brings in the private sector after the technology has been conceptualized. Successful commercialization efforts have now grown across the country, and we believe they can be described by four practical principles: 

Principle 1: A strong research enterprise is a necessary precondition to building a strong commercialization pipeline.

The first condition necessary to developing a commercialization pipeline is a reasonably advanced research enterprise. While not every region in the U.S. has access to a top-tier research university, there are pockets of excellent research at most major U.S. R1 and R2 institutions. However, because there is natural attrition at each stage of the commercialization process (much like the startup process) a critical mass of novel, leading, and relevant research activity must exist in a given University. If that bar is assumed to be the ability to attract $10 million in research funding (the equivalent of winning 20-25 SBIR Phase 1 grants annually), that limits the number of schools that can run a fruitful commercialization pipeline to approximately 350 institutions, based on data from the NSF NCSES. A metro area should have at least one research institution that meets this bar in order to secure federal funding for the development of lab-to-market programs, though given the co-location of many universities, it is possible for some metro areas to have several such research institutions or none at all.

Principle 2: Commercialization via established businesses creates different economic impacts than commercialization via startups; each pathway requires fundamentally different support.

When talking about commercialization, it is also important to differentiate between whether a new technology is brought to market by a large, incumbent company or start-up. The first half of the commercialization process is the same for both: technology is transferred out of universities, national labs, and other research institutions through the process of registering, patenting, and licensing new intellectual property (IP). Once licensed, though, the commercialization pathway branches into two. 

With an incumbent company, whether or not it successfully brings new technology to the market is largely dependent on the company’s internal goals and willingness to commit resources to commercializing that IP. Often, incumbent companies will license patents as a defensive strategy in order to prevent competition with their existing product lines. As a result, license of a technology by an incumbent company cannot be assumed to represent a guarantee of commercial use or value creation.

The alternative pathway is for universities to license their IP to start-ups, which may be spun out of university labs. Though success is not guaranteed, licensing to these new companies is where new programs and better policies can actually make an impact. Start-ups are dependent upon successful commercialization and require a lot of support to do so. Policies and programs that help meet their core needs can play a significant role in whether or not a start-up succeeds. These core needs include independent space for demonstrating and scaling their product, capital for that work and commercialization activities (e.g. scouting customers and conducting sales), and support through mentorship programs, accelerators, and in-kind help navigating regulatory processes (especially in deep tech fields). 

Principle 3: Local context matters; what works in Boston won’t necessarily work in Birmingham. 

Unfortunately, many universities approach their tech transfer programs with the goal of licensing their technology to large companies almost exclusively. This arises because university technology transfer offices (TTOs) are often understaffed, and it is easier to license multiple technologies to the same large company under an established partnership than to scout new buyers and negotiate new contracts for each patent. The Bayh-Dole Act, which established the current tech transfer system, was never intended to subsidize the R&D expenditures of our nation’s largest and most profitable companies, nor was it intended to allow incumbents to weaponize IP to repel new market entrants. Yet, that is how it is being used today in practical application.

Universities are not necessarily to blame for the lack of resources, though. Universities spend on average 0.6% of their research expenditures on their tech transfer programs. However, there is a large difference in research expenditures between top universities that can attract over a billion in research funding and the average research university, and thus a large difference in the staffing and support of TTOs. State government funding for the majority of public research universities have been declining since 2008, though there has been a slight upswing since the pandemic, while R&D funding at top universities continues to increase. Only a small minority of TTOs bring in enough income from licensing in order to be self-sustaining, often from a single “blockbuster” patent, while the majority operate at a loss to the institution. 

To successfully develop innovation capacity in ecosystems around the country through increased commercialization activity, one must recognize that communities have dramatically different levels of resources dedicated to these activities, and thus, “best practices” developed at leading universities are seldom replicable in smaller markets. 

Principle 4: Successful commercialization pipelines include interventions at the individual, institutional, and ecosystem level.

As we’ve discussed at length in our FAS “systems-thinking” blog series, which includes a post on innovation ecosystems, a systems lens is fundamental to how we see the world. Thinking in terms of systems helps us understand the structural changes that are needed to change the conditions that we see playing out around us every day. When thinking about the structure of commercialization processes, we believe that intervention at various structural levels of a system is necessary to create progres on challenges that seem insurmountable at first—such as changing the cultural expectations of “success” that are so influential in the academic systems. Below we have identified some good practices and programs for supporting commercialization at the individual, institutional, and ecosystem level, with an emphasis on pathways to start-ups and entrepreneurship.

Practices and Programs Targeted at Individuals

University tech transfer programs are often reliant on individuals taking the initiative to register new IP with their TTOs. This requires individuals to be both interested enough in commercialization and knowledgeable enough about the commercialization potential of their research to pursue registration. Universities can encourage faculty to be proactive in pursuing commercialization through recognizing entrepreneurial activities in their hiring, promotion and tenure guidelines and encouraging faculty to use their sabbaticals to pursue entrepreneurial activities. An analog to the latter at national laboratories are Entrepreneurial Leave Programs that allow staff scientists to take a leave of up to three years to start or join a start-up before returning to their position at the national lab.

Faculty and staff scientists are not the only source of IP though; graduate students and postdoctoral researchers produce much of the actual research behind new intellectual property. Whether or not these early-career researchers pursue commercialization activities is correlated with whether they have had research advisors who were engaged in commercialization. For this reason, in 2007, the National Research Foundation of Singapore established a joint research center with the Massachusetts Institute of Technology (MIT) such that by working with entrepreneurial MIT faculty members, researchers at major Singaporean universities would also develop a culture of entrepreneurship. Most universities likely can’t establish programs of this scale, but some type of mentorship program for early-career scientists pre-IP generation can help create a broader culture of translational research and technology transfer. Universities should also actively support graduate students and postdoctoral researchers in putting forward IP to their TTO. Some universities have even gone so far as to create funds to buy back the time of graduate students and postdocs from their labs and direct that time to entrepreneurial activities, such as participating in an iCorps program or conducting primary market research.  

Student at work in the NOAA CIGLR Lab at the University of Michigan School for Environment and Sustainability

Some universities have even gone so far as to create funds to buy back the time of graduate students and postdocs from their labs and direct that time to entrepreneurial activities, such as participating in an iCorps program or conducting primary market research.

Once IP has been generated and licensed, many universities offer mentorship programs for new entrepreneurs, such as MIT’s Venture Mentorship Services. Outside of universities, incubators and accelerators provide mentorship along with funding and/or co-working spaces for start-ups to grow their operation. Hardware-focused start-ups especially benefit from having a local incubator or accelerator, since hard-tech start-ups attract significantly less venture capital funding and support than digital technology start-ups, but require larger capital expenditures as they scale. Shared research facilities and testbeds are also crucial for providing hard-tech start-ups with the lab space and equipment to refine and scale their technologies.

For internationally-born entrepreneurs, an additional consideration is visa sponsorship. International graduate students and postdocs that launch start-ups need visa sponsors in order to stay in the United States as they transition out of academia. Universities that participate in the Global Entrepreneur in Residence program help provide H-1B visas for international entrepreneurs to work on their start-ups in affiliation with universities. The university benefits in return by attracting start-ups to their local community that then generate economic opportunities and help create an entrepreneurial ecosystem.

Practices and Programs Targeted at Institutions

As mentioned in the beginning, one of the biggest challenges for university tech transfer programs is understaffed TTOs and small patent budgets. On average, TTOs have only four people on staff, who can each file a handful of patents a year, and budgets for the legal fees on even fewer patents. Fully staffing TTOs can help universities ensure that new IP doesn’t slip through the cracks due to a lack of capacity for patenting or licensing activities. Developing standard term sheets for licensing agreements can also reduce administrative burden and make it easier for TTOs to establish new partnerships.

Instead of TTOs, some universities have established affiliated technology intermediaries, which are organizations that take on the business aspects of technology commercialization. For example, the Wisconsin Alumni Research Foundation (WARF) was launched as an independent, nonprofit corporation to manage the University of Wisconsin–Madison’s vitamin D patents and invest the resulting revenue into future research at the university. Since its inception 90 years ago, WARF has provided $2.3 billion in grants to the university and helped establish 60 start-up companies. 

In general, universities need to be more consistent about collecting and reporting key performance indicators for TTOs outside of the AUTM framework, such as the number of unlicensed patents and the number of products brought to the market using licensed technologies. In particular, universities should disaggregate metrics for licensing and partnerships between companies less than five years old and those greater than five years old so that stakeholders can see whether there is a difference in commercialization outcomes between incumbent and start-up licensees.

Practices and Programs Targeted at Innovation Ecosystems

Innovation ecosystems are made up of researchers, entrepreneurs, corporations, the workforce, government, and sources of capital. Geographic proximity through co-locating universities, corporations, start-ups, government research facilities, and other stakeholder institutions can help foster both formal and informal collaboration and result in significant technology-driven economic growth and benefits. Co-location may arise organically over time or result from the intentional development of research parks, such as the NASA Research Park. When done properly, the work of each stakeholder should advance a shared vision. This can create a virtuous cycle that attracts additional talent and stakeholders to the shared vision and can integrate with more traditional attraction and retention efforts. One such example is the co-location of the National Bio- and Agro-Defense Facility in Manhattan, KS, near the campus of Kansas State University. After securing that national lab, the university made investments in additional BSL-2, 3 and 3+ research facilities including the Biosecurity Research Institute and its Business Development Module. The construction and maintenance of those facilities required the creation of new workforce development programs to train HVAC technicians that manage the independent air handling capabilities of the labs and train biomanufacturing workers, which was then one of the selling points for the successful campaign for the relocation of corporation Scorpius Biologics to the region. At best, all elements of an innovation ecosystem are fueled by a research focus and the commercialization activity that it provides. 

For regions that find themselves short of the talent they need, soft-landing initiatives can help attract domestic and international entrepreneurs, start-ups, and early-stage firms to establish part of their business in a new region or to relocate entirely. This process can be daunting for early-stage companies, so soft-landing initiatives aim to provide the support and resources that will help an early-stage company acclimatize and thrive in a new place. These initiatives help to expand the reach of a community, create a talent base, and foster the conditions for future economic growth and benefits.

Alongside the creation of innovation ecosystems should be the establishment of “scale-up ecosystems” focused on developing and scaling new manufacturing processes necessary to mass produce the new technologies being developed. This is often an overlooked aspect of technology development in the United States, and supply chain shocks over the past few years have shone a light on the need to develop more local manufacturing supply chains. Fostering the growth of manufacturing alongside technology innovation can (1) reduce the time cycling between product and process development in the commercialization process, (2) capture the “learning by doing” benefits from scaling the production of new technologies, and (3) replenish the number of middle-income jobs that have been outsourced over the past few decades. 

Any way you slice it, commercialization capacity is one clear and critical input to a successful innovation ecosystem. However, it’s not the only element that’s important. A strong startup commercialization effort, standing alone, without the corporate, workforce, or government support that it needs to build a vibrant ecosystem around its entrepreneurs, might wane with time or simply be very successful at shipping spinouts off to a coastal hotspot. Building a commercialization pipeline is not, nor has it ever been, a one-size-fits-all solution for ecosystem building. 

It may even be something we’ve over-indexed on, given the widespread adoption of tech-based economic development strategies. One significant reason for this is the fact that entrepreneurship via commercialization is most open to those who already have access to a great deal of privilege–who have attained, or are on the path to, graduate degrees in STEM fields critical to our national competitiveness. If you’ve already earned a Ph.D. in machine learning, chances are your future is looking pretty bright—with or without entrepreneurial opportunity involved. To truly reap the economic benefits of commercialization activity (and the startups it creates), we need to aggressively implement programs, training, and models that change the demographics of who gets to commercialize technology, not just how they do it. To shape this, we’ll need to change the conditions for success for early-career researchers and reconsider the established model of how we mentor and train the next generation of scientists and engineers–you’ll hear more from us on these topics in future posts!

Risk and Reward in Peer Review

This article was written as a part of the FRO Forecasting project, a partnership between the Federation of American Scientists and Metaculus. This project aims to conduct a pilot study of forecasting as an approach for assessing the scientific and societal value of proposals for Focused Research Organizations. To learn more about the project, see the press release here. To participate in the pilot, you can access the public forecasting tournament here.

The United States federal government is the single largest funder of scientific research in the world. Thus, the way that science agencies like the National Science Foundation and the National Institutes of Health distribute research funding has a significant impact on the trajectory of science as a whole. Peer review is considered the gold standard for evaluating the merit of scientific research proposals, and agencies rely on peer review committees to help determine which proposals to fund. However, peer review has its own challenges. It is a difficult task to balance science agencies’ dual mission of protecting government funding from being spent on overly risky investments while also being ambitious in funding proposals that will push the frontiers of science, and research suggests that peer review may be designed more for the former rather than the latter. We at FAS are exploring innovative approaches to peer review to help tackle this challenge.

Biases in Peer Review

A frequently echoed concern across the scientific and metascientific community is that funding agencies’ current approach to peer review of science proposals tends to be overly risk-averse, leading to bias against proposals that entail high risk or high uncertainty about the outcomes. Reasons for this conservativeness include reviewer preferences for feasibility over potential impact, contagious negativity, and problems with the way that peer review scores are averaged together.

This concern, alongside studies suggesting that scientific progress is slowing down, has led to a renewed effort to experiment with new ways of conducting peer review, such as golden tickets and lottery mechanisms. While golden tickets and lottery mechanisms aim to complement traditional peer review with alternate means of making funding decisions — namely individual discretion and randomness, respectively — they don’t fundamentally change the way that peer review itself is conducted. 

Traditional peer review asks reviewers to assess research proposals based on a rubric of several criteria, which typically include potential value, novelty, feasibility, expertise, and resources. These criteria are given a score based on a numerical scale; for example, the National Institutes of Health uses a scale from 1 (best) to 9 (worst). Reviewers then provide an overall score that need not be calculated in any specific way based on the criteria scores. Next, all of the reviewers convene to discuss the proposal and submit their final overall scores, which may be different from what they submitted prior to the discussion. The final overall scores are averaged across all of the reviewers for a specific proposal. Proposals are then ranked based on their average overall score and funding is prioritized for those ranked before a certain cutoff score, though depending on the agency, some discretion by program administrators is permitted.  

The way that this process is designed allows for the biases mentioned at the beginning—reviewer preferences for feasibility, contagious negativity, and averaging problems—to influence funding decisions. First, reviewer discretion in deciding overall scores allows them to weigh feasibility more heavily than potential impact and novelty in their final scores. Second, when evaluations are discussed reviewers tend to adjust their scores to better align with their peers. This adjustment tends to be greater when correcting in the negative direction than in the positive direction, resulting in a stronger negative bias. Lastly, since funding tends to be quite limited, cutoff scores tend to be quite close to the best score. This means that even if almost all of the reviewers rate a proposal positively, one very negative review can potentially bring the average below the cutoff.

Designing a New Approach to Peer Review

In 2021, the researchers Chiara Franzoni and Paula Stephan published a working paper arguing that risk in science results from three sources of uncertainty: uncertainty of research outcomes, uncertainty of the probability of success, and uncertainty of the value of the research outcomes. To comprehensively and consistently account for these sources of uncertainty, they proposed a new expected utility approach to peer review evaluations, in which reviewers are asked to

  1. Identify the primary expected outcome of a research proposal and, optionally, a potential secondary outcome;
  2. Assess the probability between 0 to 1 of achieving each expected outcome (P(j); and
  3. Assess the value of achieving each expected outcome (uj) on a numerical scale (e.g., 0 to 100).

From this, the total expected utility can be calculated for each proposal and used to rank them.1 This systematic approach addresses the first bias we discussed by limiting the extent to which reviewers’ preferences for more feasible proposals would impact the final score of each proposal.

We at FAS see a lot of potential in Franzoni and Stephan’s expected value approach to peer review, and it inspired us to design a pilot study using a similar approach that aims to chip away at the other biases in review.

To explore potential solutions for negativity bias, we are taking a cue from forecasting by complementing the peer review process with a resolution and scoring process. This means that at a set time in the future, reviewers’ assessments will be compared to a ground truth based on the actual events that have occurred (i.e., was the outcome actually achieved and, if so, what was its actual impact?). Our theory is that if implemented in peer review, resolution and scoring could incentivize reviewers to make better, more accurate predictions over time and provide empirical estimates of a committee’s tendency to provide overly negative (or positive) assessments, thus potentially countering the effects of contagion during review panels and helping more ambitious proposals secure support. 

Additionally, we sought to design a new numerical scale for assessing the value or impact of a research proposal, which we call an impact score. Typically, peer reviewers are free to interpret the numerical scale for each criteria as they wish; Franzoni and Stephan’s design also did not specify how the numerical scale for the value of the research outcome should work. We decided to use a scale ranging from 1 (low) to 10 (high) that was base 2 exponential, meaning that a proposal that receives a score of 5 has double the impact of a proposal that receives a score of 4, and quadruple the impact of a proposal that receives a score of 3.

Plot demonstrating the exponential nature of the impact score: a score of 1 shows an impact of zero, while a score of 10 shows an impact for 1000.
Figure 1. Plot demonstrating the exponential nature of the impact score.
Table 1. Example of how to interpret the impact score.
ScoreImpact
1None or negative
2Minimal
3Low or mixed
4Moderate
5High
6Very high
7Exceptional
8Transformative
9Revolutionary
10Paradigm-shifting

The choice of an exponential scale reflects the tendency in science for a small number of research projects to have an outsized impact (Figure 2), and provides more room at the top end of the scale for reviewers to increase the rating of the proposals that they believe will have an exceptional impact. We believe that this could help address the last bias we discussed, which is that currently, bad scores are more likely to pull a proposal’s average below the cutoff than good scores are likely to pull a proposal’s average above the cutoff.

Figure 2. Citation distribution of accepted and rejected journal articles

We are now piloting this approach on a series of proposals in the life sciences that we have collected for Focused Research Organizations, a new type of non-profit research organization designed to tackle challenges that neither academia or industry is incentivized to work on. The pilot study was developed in collaboration with Metaculus, a forecasting platform and aggregator, and will be hosted on their website. We welcome subject matter experts in the life sciences — or anyone interested! — to participate in making forecasts on these proposals here. Stay tuned for the results of this pilot, which we will publish in a report early next year.

Collaboration for the Future of Public and Active Transportation

Summary

Public and active transportation are not equally accessible to all Americans. Due to a lack of sufficient infrastructure and reliable service for public transportation and active modes like biking, walking, and rolling, Americans must often depend on personal vehicles for travel to work, school, and other activities. During the past two years, Congress has allocated billions of dollars to equitable infrastructure, public transportation upgrades, and decreasing greenhouse gas pollution from transportation across the United States. The Department of Transportation (DOT) and its agencies should embrace innovation and partnerships to continue to increase active and public transportation across the country. The DOT should require grant applications for funding to discuss cross-agency collaborations, partner with the Department of Housing and Urban Development (HUD) to organize prize competitions, encourage public-private partnerships (P3s), and work with the Environmental Protection Agency (EPA) to grant money for transit programs through the Greenhouse Gas Reduction Fund. 

Challenge and Opportunity

Historically, U.S. investment in transportation has focused on expanding and developing highways for personal vehicle travel. As a result, 45% of Americans do not have access to reliable and safe public transportation, perpetuating the need for single-use vehicles for almost half of the country. The EPA reports that transportation accounts for 29% of total U.S. greenhouse gas emissions, with 58% of those emissions coming from light-duty cars. This large share of nationwide emissions from personal vehicles has short- and long-term climate impacts. 

Investments in green public and active transit should be a priority for the DOT in transitioning away from a personal-vehicle-dominated society and meeting the Biden Administration’s “goals of a 100% clean electrical grid by 2035 and net-zero carbon emissions by 2050.” Public and active transportation infrastructure includes bus systems, light rail, bus rapid transit, bike lanes, and safe sidewalks. Investments in public and active transportation should go towards a combination of electrifying existing public transportation, such as buses; improving and expanding public transit to be more reliable and accessible for more users; constructing bike lanes; developing community-owned bike share programs; and creating safe walking corridors. 

In addition to reducing carbon emissions, improved public transportation that disincentivizes personal vehicle use has a variety of co-benefits. Prioritizing public and active transportation could limit congestion on roads and lower pollution. Fewer vehicles on the road result in less tailpipe emissions, which “can trigger health problems such as aggravated asthma, reduced lung capacity, and increased susceptibility to respiratory illnesses, including pneumonia and bronchitis.” This is especially important for the millions of people who live near freeways and heavily congested roads. 

Congestion can also be financially costly for American households; the INRIZ Global Traffic Scorecard reports that traffic congestion cost the United States $81 billion in 2022. Those costs include vehicle maintenance, fuel cost, and “lost time,” all of which can be reduced with reliable and accessible public and active transportation. Additionally, the American Public Transportation Association reports that every $1 invested in public transportation generates $5 in economic returns, measured by savings in time traveled, reduction in traffic congestion, and business productivity. Thus, by investing in public transportation, communities can see improvements in air quality, economy, and health.

Public transportation is primarily managed at the local and state level; currently, over  6000 local and state transportation agencies provide and oversee public transportation in their regions. Public transportation is funded through federal, state, and local sources, and transit agencies receive funding from “passenger fares and other operating receipts.” The Federal Transit Administration (FTA) distributes funding for transit through grants and loans and accounts for 15% of total income for transit agencies, including 31% of capital investments in transit infrastructure. Local and state entities often lack sufficient resources to improve public transportation systems because of the uncertainty of ridership and funding streams.

Public-private partnerships can help alleviate some of these resource constraints because contracts can allow the private partner to operate public transportation systems. Regional and national collaboration across multiple agencies from the federal to the municipal level can also help alleviate resource barriers to public transit development. Local and state agencies do not have to work alone to improve public and active transportation systems. 

The following recommendations provide a pathway for transportation agencies at all levels of government to increase public and active transportation, resulting in social, economic, and environmental benefits for the communities they serve. 

Plan of Action

Recommendation 1. The FTA should require grant applicants for programs such as the Rebuilding American Infrastructure with Sustainability and Equity (RAISE) to define how they will work collaboratively with multiple federal agencies and conduct community engagement. 

Per the National Blueprint for Transportation Decarbonization, FTA staff should prioritize funding for grant applicants who successfully demonstrate partnerships and collaboration. This can be demonstrated, for example, with letters of support from community members and organizations for transit infrastructure projects. Collaboration can also be demonstrated by having applicants report clear goals, roles, and responsibilities for each agency involved in proposed projects. The FTA should: 

  1. Develop a rubric for evaluating partnerships’ efficiency and alignment with national transit decarbonization goals. 
  2. Create a tiered metrics system within the rubric that prioritizes grants for projects based on collaboration and reduction of greenhouse gas emissions in the transit sector.
  3. Add a category to their Guidance Center on federal-state-local partnerships to provide insight on how they view successful collaboration. 

Recommendation 2. The DOT and HUD should collaborate on a prize competition to design active and/or public transportation projects to reduce traffic congestion. 

Housing and transportation costs are related and influence one another, which is why HUD is a natural partner. Funding can be sourced from the Highway Trust Fund, which the DOT has the authority to allocate up to “1% of the funds for research and development to carry out . . . prize competition program[s].”

This challenge should call on local agency partners to provide a design challenge or opportunity that impedes their ability to adopt transit-oriented infrastructure that could reduce traffic congestion. Three design challenges should be selected and publicly posted on the Challenge.gov website so that any individual or organization can participate. 

The goal of the prize competition is to identify challenges, collaborate, and share resources across agencies and communities to design transportation solutions. The competition would connect the DOT with local and regional planning and transportation agencies to solicit solutions from the public, whether from individuals, teams of individuals, or organizations. The DOT and HUD should work collaboratively to design the selection criteria for the challenge and select the winners. Each challenge winner would be provided with a financial prize of $250,000, and their idea would be housed on the DOT website as a case study that can be used for future planning decisions. The local agencies that provide the three design challenges would be welcome to implement the winning solutions.

Recommendation 3. Federal, state, and local government should increase opportunities for public-private partnerships (P3s). 

The financial investment required to develop active and public transportation infrastructure is a hurdle for many agencies. To address this issue, we make the following recommendations: 

Conclusion

The road to decarbonizing the transportation sector requires public and active transportation. Federal agencies can allocate funding for public and active transit more effectively through the recommendations above. It’s time for the government to recognize public and active transportation as the key to equitable decarbonization of the transportation sector throughout the United States.

Frequently Asked Questions
What are examples of transit public-private partnerships (P3s)?

Most P3s in the United States are for highways, bridges, and roads, but there have been a few successful public transit P3s. In 2018 the City of Los Angeles joined LAX and LAX Integrated Express Solutions in a $4.9 billion P3 to develop a train system within the airport. This project aims to launch in 2024 to “enhance the traveler experience” and will “result in 117,000 fewer vehicle miles traveled per day” to the airport. This project is a prime example of how P3s can help reduce traffic congestion and enable and encourage the use of public transportation.

How can P3s be further supported through federal policy beyond the recommendations in this memo?

In 2021, the Congressional Research Service released a report about public-private partnerships (3Ps) that highlights the role the federal government can play by making it easier for agencies to participate in P3s.

What are examples of existing green banks and infrastructure banks?

The state of Michigan has a long history with its Michigan Saves program, the nation’s first nonprofit green bank, which provides funding for projects like rooftop solar or energy efficiency programs.


In California the California Alternative Energy and Advanced Transportation Financing Authority works “collaboratively with public and private partners to provide innovative and effective financing solutions” for renewable energy sources, energy efficiency, and advanced transportation and manufacturing technologies.


The Rhode Island Infrastructure Bank provides funding to municipalities, businesses, and homeowners for projects “including water and wastewater, roads and bridges, energy efficiency and renewable energy, and brownfield remediation.”

Applying ARPA-I: A Proven Model for Transportation Infrastructure

Executive Summary

In November 2021, Congress passed the Infrastructure Investment and Jobs Act (IIJA), which included $550 billion in new funding for dozens of new programs across the U.S. Department of Transportation (USDOT). Alongside historic investments in America’s roads and bridges, the bill created the Advanced Research Projects Agency-Infrastructure (ARPA-I). Building on successful models like the Defense Advanced Research Projects Agency (DARPA) and the Advanced Research Program-Energy (ARPA-E), ARPA-I’s mission is to bring the nation’s most innovative technology solutions to bear on our most significant transportation infrastructure challenges.

ARPA-I must navigate America’s uniquely complex infrastructure landscape, characterized by limited federal research and development funding compared to other sectors, public sector ownership and stewardship, and highly fragmented and often overlapping ownership structures that include cities, counties, states, federal agencies, the private sector, and quasi-public agencies. Moreover, the new agency needs to integrate the strong culture, structures, and rigorous ideation process that ARPAs across government have honed since the 1950s. This report is a primer on how ARPA-I, and its stakeholders, can leverage this unique opportunity to drive real, sustainable, and lasting change in America’s transportation infrastructure.

How to Use This Report

This report highlights the opportunity ARPA-I presents; orients those unfamiliar with the transportation infrastructure sector to the unique challenges it faces; provides a foundational understanding of the ARPA model and its early-stage program design; and empowers experts and stakeholders to get involved in program ideation. However, individual sections can be used as standalone tools depending on the reader’s prior knowledge of and intended involvement with ARPA-I.

An Opportunity for Transportation Infrastructure Innovation

In November 2021, Congress passed the Infrastructure Investment and Jobs Act (IIJA) authorizing the U.S. Department of Transportation (USDOT) to create the Advanced Research Projects Agency-Infrastructure (ARPA-I), among other new programs. ARPA-I’s mission is to advance U.S. transportation infrastructure by developing innovative science and technology solutions that:

ARPA-I will achieve this goal by supporting research projects that:

ARPA-I is the newest addition to a long line of successful ARPAs that continue to deliver breakthrough innovations across the defense, intelligence, energy, and health sectors. The U.S. Department of Defense established the pioneering Defense Advanced Research Projects Agency (DARPA) in 1958 in response to the Soviet launch of the Sputnik satellite to develop and demonstrate high-risk, high-reward technologies and capabilities to ensure U.S. military technological superiority and confront national security challenges. Throughout the years, DARPA programs have been responsible for significant technological advances with implications beyond defense and national security, such as the early stages of the internet, the creation of the global positioning system (GPS), and the development of mRNA vaccines critical to combating COVID-19. 

In light of the many successful advancements seeded through DARPA programs, the government replicated the ARPA model for other critical sectors, resulting in the Intelligence Advanced Research Projects Activity (IARPA) within the Office of the Director of National Intelligence, the Advanced Research Projects Agency-Energy within the Department of Energy, and, most recently, the Advanced Research Projects Agency-Health (ARPA-H) within the Department of Health and Human Services.

Now, there is the opportunity to bring that same spirit of untethered innovation to solve the most pressing transportation infrastructure challenges of our time. The United States has long faced a variety of transportation infrastructure-related challenges, due in part to low levels of federal research and development (R&D) spending in this area; the fragmentation of roles across federal, state, and local government; risk-averse procurement practices; and sluggish commercial markets. These challenges include:

The Fiscal Year 2023 Omnibus Appropriations Bill awarded ARPA-I its initial appropriation in early 2023. Yet even before that, the Biden-Harris Administration saw the potential for ARPA-I-driven innovations to help meet its goal of net-zero GHG emissions by 2050, as articulated in its Net-Zero Game Changers Initiative. In particular, the Administration identified smart mobility, clean and efficient transportation systems, next-generation infrastructure construction, advanced electricity infrastructure, and clean fuel infrastructure as “net-zero game changers” that ARPA-I could play an outsize role in helping develop.

For ARPA-I programs to reach their full potential, agency stakeholders and partners need to understand not only how to effectively apply the ARPA model but how the unique circumstances and challenges within transportation infrastructure need to be considered in program design.

Unique Challenges of the Transportation Infrastructure Landscape

Using ARPA-I to advance transportation infrastructure breakthroughs requires an awareness of the most persistent challenges to prioritize and the unique set of circumstances within the sector that can hinder progress if ignored. Below are summaries of key challenges and considerations for ARPA-I to account for, followed by a deeper analysis of each challenge.

Lower Federal R&D Spending in Transportation Infrastructure 

Federal R&D expenditures in transportation infrastructure lag behind those in other sectors. This gap is particularly acute because, unlike for some other sectors, federal transportation R&D expenditures often fund studies and systems used to make regulatory decisions rather than technological innovation. The table below compares actual federal R&D spending and sector expenditures for 2019 across defense, healthcare, energy, and transportation as a percentage of each sector’s GDP. The federal government spends orders of magnitude less on transportation than other sectors: energy R&D spending as a percentage of sector GDP is nearly 15 times higher than transportation, while health is 13 times higher and defense is nearly 38 times higher.

Agency ^1Actual federal R&D spending, 2019Value added by industry and % of U.S. GDP, 20192019 federal R&D spending as % of sector GDP
Defense$54.69 billion$732 billion (3.4%)7.5%
Health and Human Services$38.51 billion$1,452 billion (6.8%) ^22.7%
Energy$18.27 billion$607 billion (2.8%) ^33.0%
Transportation$1.10 billion$610 billion (2.9%) ^40.2%
Expand Footnotes

1. The comparison of federal R&D spending and sector expenditures for 2019 is similar to those for the years 2020 and 2021.

2. Excludes GDP value-adds relating to Social Assistance

3. Includes GDP value-adds relating to oil and gas extraction, utilities, and petroleum and coal products

4. Excludes GDP value adds relating to Warehousing

Public Sector Dominance Limits Innovation Investment 

Since 1990, total investment in U.S. R&D has increased by roughly 9 times. When looking at the source of R&D investment over the same period, the private and public sectors invested approximately the same amount of R&D funding in 1982, but today the rate of R&D investment is nearly 4 times greater for the private industry than the government. 

While there are problems with the bulk of R&D coming from the private sector, such as innovations to promote long-term public goods being overlooked because of more lucrative market incentives, industries that receive considerable private R&D funding still see significant innovation breakthroughs. For example, the medical industry saw $161.8 billion in private R&D funding in 2020 compared to only $61.5 billion from federal funding. More than 75% of this private industry R&D occurred within the biopharmaceutical sector where corporations have profit incentives to be at the cutting edge of advancements in medicine.

The transportation sector has one robust domain for private R&D investment: vehicle and aircraft equipment manufacturing. In 2018, total private R&D was $52.6 billion. Private sector transportation R&D focuses on individual customers and end users, creating better vehicles, products, and efficiencies. The vast majority of that private sector R&D does not go toward infrastructure because the benefits are largely public rather than private. Put another way, the United States invests more than 50 times the amount of R&D into vehicles than the infrastructure systems upon which those vehicles operate. 

Market Fragmentation across Levels of Government

Despite opportunities within the public-dominated transportation infrastructure system, market fragmentation is a persistent obstacle to rapid progress. Each level of government has different actors with different objectives and responsibilities. For instance, at the federal level, USDOT provides national-level guidance, policy, and funding for transportation across aviation, highway, rail, transit, ports, and maritime modes. Meanwhile, the states set goals, develop transportation plans and projects, and manage transportation networks like the interstate highway system. Metropolitan planning organizations take on some of the planning functions at the regional level, and local governments often maintain much of their infrastructure. There are also local individual agencies that operate facilities like airports, ports, or tollways organized at the state, regional, or local level. Programs that can use partnerships to cut across this tapestry of systems are essential to driving impact at scale. 

Local agencies have limited access and capabilities to develop cross-sector technologies. They have access to limited pools of USDOT funding to pilot technologies and thus generally rely on commercially available technologies to increase the likelihood of pilot success. One shortcoming of this current process is that both USDOT and infrastructure owner-operators (IOOs) play a more passive role in developing innovative technologies, instead depending on merely deploying market-ready technologies. 

Multiple Modes, Customers, and Jurisdictions Create Difficulties in Efficiently Allocating R&D Resources

The transportation infrastructure sector is a multimodal environment split across many modes, including aviation, maritime, pipelines, railroads, roadways (which includes biking and walking), and transit. Each mode includes various customers and stakeholders to be considered. In addition, in the fragmented market landscape federal, state, and local departments of transportation have different—and sometimes competing—priorities and mandates. This dynamic creates difficulties in allocating R&D resources and considering access to innovation across these different modes.

Customer identification is not “one size fits all” across existing ARPAs. For example, DARPA has a laser focus on delivering efficient innovations for one customer: the Department of Defense. For ARPA-E, it is less clear; their customers range from utility companies to homeowners looking to benefit from lower energy costs. ARPA-I would occupy a space in between these two cases, understanding that its end users are IOOs—entities responsible for deploying infrastructure in many cases at the local or regional level. 

However, even with this more direct understanding of its customers, a shortcoming of a system focused on multiple modes is that transportation infrastructure is very broad, occupying everything from self-healing concrete to intersection safety to the deployment of electrified mobility and more. Further complicating matters is the rapid evolution of technologies and expectations across all modes, along with the rollout of entirely new modes of transportation. These developments raise questions about where new technologies and capabilities fit in existing modal frameworks, what actors in the transportation infrastructure market should lead their development, and who the ultimate “customers” or end users of innovation are.

Having a matrixed understanding of the rapid technological evolution across transportation modes and their potential customers is critical to investing in and building infrastructure for the future, given that transportation infrastructure investments not only alter a region’s movement of people and goods but also fundamentally impact its development. ARPA-I is poised to shape learnings across and in partnership with USDOT’s modes and various offices to ensure the development and refinement of underlying technologies and approaches that serve the needs of the entire transportation system and users across all modes.

Core Tenets of ARPA Success

Success using the ARPA model comes from demonstrating new innovative capabilities, building a community of people (an “ecosystem”) to carry the progress forward, and having the support of key decision-makers. Yet the ARPA model can only be successful if its program directors (PDs), fellows, stakeholders, and other partners understand the unique structure and inherent flexibility required when working to create a culture conducive to spurring breakthrough innovations. From a structural and cultural standpoint, the ARPA model is unlike any other agency model within the federal government, including all existing R&D agencies. Partners and other stakeholders should embrace the unique characteristics of an ARPA.

Cultural Components

ARPAs should take risks. 

An ARPA portfolio may be the closest thing to a venture capital portfolio in the federal government. They have a mandate to take big swings so should not be limited to projects that seem like safe bets. ARPAs will take on many projects throughout their existence, so they should balance quick wins with longer-term bets while embracing failure as a natural part of the process.

ARPAs should constantly evaluate and pivot when necessary.

An ARPA needs to be ruthless in its decision-making process because it has the ability to maneuver and shift without the restriction of initial plans or roadmaps. For example, projects around more nascent technology may require more patience, but if assessments indicate they are not achieving intended outcomes or milestones, PDs should not be afraid to terminate those projects and focus on other new ideas.

ARPAs should stay above the political fray. 

ARPAs can consider new and nontraditional ways to fund innovation, and thus should not be caught up in trends within their broader agency. As different administrations onboard, new offices get built and partisan priorities may shift, but ARPAs should limit external influence on their day-to-day operations.

ARPA team members should embrace an entrepreneurial mindset. 

PDs, partners, and other team members need to embrace the creative freedom required for success and operate much like entrepreneurs for their programs. Valued traits include a propensity toward action, flexibility, visionary leadership, self-motivation, and tenacity.

ARPA team members must move quickly and nimbly.

Trying to plan out the agency’s path for the next two years, five years, 10 years, or beyond is a futile effort and can be detrimental to progress. ARPAs require ultimate flexibility from day to day and year to year. Compared to other federal initiatives, ARPAs are far less bureaucratic by design, and forcing unnecessary planning and bureaucracy on the agency will slow progress.

Collegiality must be woven into the agency’s fabric.

With the rapidly shifting and entrepreneurial nature of ARPA work, the federal staff, contractors, and other agency partners need to rely on one another for support and assistance to seize opportunities and continue progressing as programs mature and shift. 

Outcomes matter more than following a process.

ARPA PDs must be free to explore potential program and project ideas without any predetermination. The agency should support them in pursuing big and unconventional ideas unrestricted by a particular process. While there is a process to turn their most unconventional and groundbreaking ideas into funded and functional projects, transformational ideas are more important than the process itself during idea generation.

ARPA team members welcome feedback.

Things move quickly in an ARPA, and decisions must match that pace, so individuals such as fellows and PDs must work together to offer as much feedback as possible. Constructive pushback helps avoid blind alleys and thus makes programs stronger.

Structural Components

The ARPA Director sets the vision.

The Director’s vision helps attract the right talent and appropriate levels of ambition and focus areas while garnering support from key decision-makers and luminaries. This vision will dictate the types and qualities of PDs an ARPA will attract to execute within that vision.

PDs can make or break an ARPA and set the technical direction.

Because the power of the agency lies within its people, ARPAs are typically flat organizations. An ARPA should seek to hire the best and most visionary thinkers and builders as PDs, enable them to determine and design good programs, and execute with limited hierarchical disruption. During this process, PDs should engage with decision-makers in the early stages of the program design to understand the needs and realities of implementers.

Contracting helps achieve goals.

The ARPA model allows PDs to connect with universities, companies, nonprofits, organizations, and other areas of government to contract necessary R&D. This allows the program to build relationships with individuals without needing to hire or provide facilities or research laboratories. 

Interactions improve outcomes. 

From past versions of ARPA that attempted remote and hybrid environments, it became evident that having organic collisions across an ARPA’s various roles and programs is important to achieving better outcomes. For example, ongoing in-person interactions between and among PDs and technical advisors are critical to idea generation and technical project and program management. 

Staff transitions must be well facilitated to retain institutional knowledge. 

One of ARPA’s most unique structural characteristics is its frequent turnover. PDs and fellows are term-limited, and ARPAs are designed to turn over those key positions every few years as markets and industries evolve, so having thoughtful transition processes in place is vital, including considering the role of systems engineering and technical assistance (SETA) contractors in filling knowledge gaps, cultivating an active alumni network, and staggered hiring cycles so that large numbers of PDs and fellows are not all exiting their service at once.

Scaling should be built into the structure.

It cannot be assumed that if a project is successful, the private sector will pick that technology up and help it scale. Instead, an ARPA should create its own bridge to scaling in the form of programs dedicated to funding projects proven in a test environment to scale their technology for real-world application. 

Technology-to-market advisors play a pivotal role.

Similarly to the dedicated funding for scaling described above, technology-to-market advisors are responsible for thinking about how projects make it to the real world. They should work hand in hand with PDs even in the early stages of program development to provide perspectives on how projects might commercialize and become market-ready. Without this focus, technologies run the risk of dying on the vine—succeeding technically, but failing commercially. 

A Primer on ARPA Ideation

Tackling grand challenges in transportation infrastructure through ARPA-I requires understanding what is unique about its program design. This process begins with considering the problem worth solving, the opportunity that makes it a ripe problem to solve, a high-level idea of an ARPA program’s fit in solving it, and a visualization of the future once this problem has been solved. This process of early-stage program ideation requires a shift in one’s thinking to find ideas for innovative programs that fit the ARPA model in terms of appropriate ambition level and suitability for ARPA structure and objectives. It is also an inherently iterative process, so while creating a “wireframe” outlining the problem, opportunity, program objectives, and future vision may seem straightforward, it can take months of refining. 

Common Challenges

No clear diagnosis of the problem

Many challenges facing our transportation infrastructure system are not defined by a single problem; rather, they are a conglomeration of issues that simultaneously need addressing. An effective program will not only isolate a single problem to tackle, but it will approach it at a level where something can be done to solve it through root cause analysis.

Thinking small and narrow

On the other hand, problems being considered for ARPA programs can be isolated down to the point that solving them will not drive transformational change. In this situation, narrow problems would not cater to a series of progressive and complementary projects that would fit an ARPA.

Incorrect framing of opportunities:

When doing early-stage program design, opportunities are sometimes framed as “an opportunity to tackle a problem.” Rather, an opportunity should reflect a promising method, technology, or approach already in existence but which would benefit from funding and resources through an advanced research agency program.

Approaching solutions solely from a regulatory or policy angle

While regulations and policy changes are a necessary and important component of tackling challenges in transportation infrastructure, approaching issues through this lens is not the mandate of an ARPA. ARPAs focus on supporting breakthrough innovations in developing new methods, technologies, capabilities, and approaches. Additionally, regulatory approaches to problem-solving can often be subject to lengthy policy processes.

No explicit ARPA role

An ARPA should pursue opportunities to solve problems where, without its intervention, breakthroughs may not happen within a reasonable timeframe. If the public or private sector already has significant interest in solving a problem, and they are well on their way to developing a transformational solution in a few years or less, then ARPA funding and support might provide a higher value-add elsewhere.

Lack of throughline

The problems identified for ARPA program consideration should be present as themes throughout the opportunities chosen to solve them as well as how programs are ultimately structured. Otherwise, a program may lack a targeted approach to solving a particular challenge.

Forgetting about end users

Human-centered design should be at the heart of how ARPA programs are scoped, especially when considering the scale at which designers need to think about how solving a problem will provide transformational change for everyday users.

Being solutions-oriented

Research programs should not be built with predetermined solutions in mind; they should be oriented around a specific problem to ensure that any solutions put forward are targeted and effective.

Not being realistic about direct outcomes of the program

Program objectives should not simply restate the opportunity, nor should they jump to where the world will be many years after the program has run its course. They should separate the tactical elements of a program and what impact they will ultimately drive. Designers should consider their program as one key step in a long arc of commercialization and adoption, with a firm sense of who needs to act and what needs to happen to make a program objective a reality.

Keeping these common mistakes in mind throughout the design process ensures that programs are properly scoped, appropriately ambitious, and in line with the agency’s goals. With these guideposts in mind, idea generators should begin their program design in the form of a wireframe.

Wireframe Development 

The first phase in ARPA program development is creating a program wireframe, which is an outline of a potential program that captures key components for consideration to assess the program’s fit and potential impact. The template below shows the components characteristic of a program wireframe.

Wireframe template

To create a fully fleshed-out wireframe, program directors work backward by first envisioning a future state that would be truly transformational for society and across sectors if it were to be realized. Then, they identify a clearly-articulated problem that needs solving and is hindering progress toward this transformational future state. During this process, PDs need to conduct extensive root cause analysis to consider whether the problem they’ve identified is exacerbated by policy, regulatory, or environmental complications—as opposed to those that technology can already solve. This will inform whether a problem is something that ARPA-I has the opportunity to impact fundamentally. 

Next, program directors identify a promising opportunity—such as a method, approach, or technology—that, if developed, scaled, and implemented, would solve the problem they articulated and help achieve their proposed future state. When considering a promising opportunity, PDs must assess whether it front-runs other potential technologies that would also need developing to support it and whether it is feasible to achieve concrete results within three to five years and with an average program budget. Additionally, it is useful to think about whether an opportunity considered for program development is part of a larger cohort of potential programs that lie within an ARPA-I focus area that could all be run in parallel.

Most importantly, before diving into how to solve the problem, PDs need to articulate what has prevented this opportunity from already being solved, scaled, and implemented, and what explicit role or need there is for a federal R&D agency to step in and lead the development of technologies, methods, or approaches to incentivize private sector deployment and scaling. For example, if the private sector is already incentivized to, and capable of, taking the lead on developing a particular technology and it will achieve market readiness within a few years, then there is less justification for an ARPA intervention in that particular case. On the other hand, the prescribed solution to the identified problem may be so nascent that what is needed is more early-stage foundational R&D, in which case an ARPA program would not be a good fit. This area should be reserved as the domain of more fundamental science-based federal R&D agencies and offices.

One example to illustrate this maturity fit is DARPA investment in mRNA. While the National Institutes of Health contributed significantly to initial basic research, DARPA recognized the technological gap in being able to quickly scale and manufacture therapeutics, prompting the agency to launch the Autonomous Diagnostics to Enable Prevention and Therapeutics (ADEPT) program to develop technologies to respond to infectious disease threats. Through ADEPT, in 2011 DARPA awarded a fledgling Moderna Therapeutics with $25 million to research and develop its messenger RNA therapeutics platform. Nine years later, Moderna became the second company after Pfizer-BioNTech to receive an Emergency Use Authorization for its COVID-19 vaccine.

Another example is DARPA’s role in developing the internet as we know it, which was not originally about realizing the unprecedented concept of a ubiquitous, global communications network. What began as researching technologies for interlinking packet networks led to the development of ARPANET, a pioneering network for sharing information among geographically separated computers. DARPA then contracted BBN Technologies to build the first routers before becoming operational in 1969. This research laid the foundation for the internet. The commercial sector has since adopted ARPANET’s groundbreaking results and used them to revolutionize communication and information sharing across the globe.

Wireframe Refinement and Iteration

To guide program directors through successful program development, George H. Heilmeier, who served as the director of DARPA from 1975 to 1977, used to require that all PDs answer the following questions, known as the Heilmeier Catechism, as part of their pitch for a new program. These questions should be used to refine the wireframe and envision what the program could look like. In particular, wireframe refinement should examine the first three questions before expanding to the remaining questions.

Alongside the Heilmeier Catechism, a series of assessments and lines of questioning should be completed to pressure test and iterate once the wireframe has been drafted. This refinement process is not one-size-fits-all but consistently grounded in research, discussions with experts, and constant questioning to ensure program fit. The objective is to thoroughly analyze whether the problem we are seeking to solve is the right one and whether the full space of opportunities around that problem is ripe for ARPA intervention.

One way to think about determining whether a wireframe could be a program is by asking, “Is this wireframe science or is this science fiction?” In other words, is the proposed technology solution at the right maturity level for an ARPA to make it a reality? There is a relatively broad range in the middle of the technological maturity spectrum that could be an ARPA program fit, but the extreme ends of that spectrum would not be a good fit, and thus those wireframes would need further iteration or rejection. On the far left end of the spectrum would be basic research that only yields published papers or possibly a prototype. On the other extreme would be a technology that is already developed to the point that only full-scale implementation is needed. Everything that falls between could be suitable for an ARPA program topic area.  

Once a high-impact program has been designed, the next step is to rigorously pressure test and develop a program until it resembles an executable ARPA program.

Applying ARPA Frameworks to Transportation Infrastructure Challenges

By using this framework, any problem or opportunity within transportation infrastructure can be evaluated for its fit as an ARPA-level idea. Expert and stakeholder idea generation is essential to creating an effective portfolio of ARPA-I programs, so idea generators must be armed with this framework and a defined set of focus areas to develop promising program wireframes. An initial set of focus areas for ARPA-I includes safety, climate and resilience, and digitalization, with equity and accessibility as underlying considerations within each focus area. 

There are hundreds of potential topic areas that ARPA-I could tackle; the two wireframes below represent examples of early-stage program ideas that would benefit from further pressure testing through the program design iteration cycle.

Note: The following wireframes are samples intended to illustrate ARPA ideation and the wireframing process, and do not represent potential research programs or topics under consideration by the U.S. Department of Transportation.

Next-Generation Resilient Infrastructure Management

A Digital Inventory of Physical Infrastructure and Its Uses

Wireframe Development Next Steps

After initial wireframe development, further exploration is needed to pressure test an idea and ensure that it can be developed into a viable program to achieve “moonshot” ambitions. Wireframe authors should consider the following factors when iterating:

Wireframes are intended to be a summary communicative of a larger plan to follow. After further iteration and exploration of the factors outlined above, what was first just a raw program wireframe should develop into more detailed documents. These should include an incisive diagnosis of the problem and evidence and citations validating opportunities to solve it. Together, these components should lead to a plausible program objective as an outcome.

Conclusion

The newly authorized and appropriated ARPA-I presents a once-in-a-generation opportunity to apply a model that has been proven successful in developing breakthrough innovations in other sectors to the persistent challenges facing transportation infrastructure.

Individuals and organizations that would work within the ARPA-I network need to have a clear understanding of the unique circumstances, challenges, and opportunities of this sector, as well as how to apply this context and the unique ARPA program ideation model to build high-impact future innovation programs. This community’s engagement is critical to ARPA-I’s success, and the FAS is looking for big thinkers who are willing to take on this challenge by developing bold, innovative ideas.

To sign up for future updates on events, convenings, and other opportunities for you to work in support of ARPA-I programs and partners, click here.

To submit an advanced research program idea, click here.

Proposed House Budget Would Reduce Federal R&D By An Estimated $442 Billion or 19% Over 10 Years

On Wednesday, April 19, Speaker McCarthy unveiled the Limit, Save, Grow Act of 2023, which would establish a set of discretionary spending caps over the next decade through FY 2033, allowing for only sub-inflation increases in overall spending. These caps would have the effect of reducing base discretionary spending by over $3.5 trillion below baseline over that time.

Read the full analysis here.

The COVID Emergency is Almost Over—What Did We Learn About Rapid R&D?

This month marks three years since the COVID pandemic took hold of nearly every aspect of American life. In a few more months (May 2023) the coronavirus public health emergency is set to conclude officially, following the Administration’s announcement to wind down the declarations. As the nation grapples with the tragedies and lasting effects of the pandemic, it should also take lessons from the most successful elements of how the government responded to the challenge. The most notable success might be Operation Warp Speed (OWS), the highly successful public-private partnership that produced and distributed millions of live-saving vaccines in record time. Our new memo, How to Replicate the Success of Operation Warp Speed, helps this audience assess how and if they even should attempt to replicate the approach.

The Federation of American Scientists, together with our partners at 1Day Sooner and the Institute for Progress (IFP), convened leadership from the original OWS team, agency heads, Congressional staffers, researchers, and representatives from vaccine manufacturers in November 2022 to reflect on the success of the program and future applications of the model. The memo was developed primarily from notes on presentations, panel discussions, and lively breakout conversations that were both reflective and forward looking. This piece complements other analyses by providing a practical, playbook-style approach. Those looking to replicate the success of OWS should consider the stakeholder landscape and the state of fundamental science before designing a portfolio of policy interventions. 

Assess the stakeholder landscape and science surrounding the challenge

A program on the exact scale of OWS will only work for major national challenges that are self-evidently important and urgent. Designers should assess the stakeholder landscape and consider the political, regulatory, and behavioral contexts. The fundamental research must exist, and the goal should require advancing it for a specific use case at scale. Technology readiness levels (TRLs) can help guide technology assessment—a level of at least 5 is a good bet. All decisions about technology readiness should be made using the best available science, data, and demonstrated capabilities.

Design an agile program by selecting a portfolio of interventions

Choose a selection of the mechanisms below informed by the stakeholder and technology assessment. The organization of R&D, manufacturing, and deployment should be inspired by agile methodology, in which planning is iterative and more risk than normal is accepted. 

  1. Establish a leadership team across federal agencies
  2. Coordinate federal agencies and the private sector
  3. Activate latent private-sector capacities for labor and manufacturing
  4. Shape markets with demand-pull mechanisms
  5. Reduce risk with diversity and redundancy

Operation Warp Speed was a historic accomplishment on the level of the Manhattan Project and the Apollo program, but the unique approach is not appropriate for everything. Read the full memo to understand the mechanisms and the types of challenges best suited for this approach. Even if a challenge does not meet the criteria for the full OWS treatment, the five mechanisms can be applied individually to better coordinate agencies and the private sector toward solutions.

ALI Releases Statement on the President’s FY2024

WASHINGTON, D.C. — The Alliance for Learning Innovation (ALI) applauds the increases proposed for education research and development (R&D) and innovation in the President’s budget request. These include the $870.9 million proposed for the Institute of Education Sciences (IES), including $75 million for a National Center for Advanced Development in Education (NCADE), the $405 million proposed for the Education Innovation and Research (EIR) program and the $1.4 billion for the National Science Foundation’s (NSF) Directorate for STEM Education. These investments represent real commitments to advancing an inclusive education research system that centers students, teachers, and communities.

These recommendations build upon the bipartisan interest in utilizing education R&D to  accelerate learning recovery, increase student achievement, and ensure students and teachers are prepared for the continued impact technology will have on teaching and learning. National and economic security depends on the success of our students and ALI appreciates the priorities this budget request places on fostering innovations in education that will support U.S. competitiveness.

Dan Correa, CEO of the Federation of American Scientists and co-lead of ALI notes, “Investments in education research and development hold so much promise for dramatically improving gaps in student achievement. Learning recovery, workforce development, and global competition all demand a pool of talent that can only come from an education system that meets the needs of diverse learners. The President’s budget request recognizes that more robust education R&D is needed to support bold innovations that meet the needs of students, teachers, families, and communities.”

This budget will allow IES and other federal agencies the ability to build on boundary-pushing efforts like the National AI Institute for Exceptional Education, which is supporting advancements in AI, human-AI interaction, and learning science to improve educational outcomes for children with speech and language related challenges.

For too long, federal support for education R&D has languished while resources and attention have been devoted to R&D in health care, defense, energy, and other fields. Today’s budget represents a critical step forward in addressing this deficiency. The Alliance for Learning Innovation looks forward to championing the continued development of an education R&D ecosystem that will lead to the types of groundbreaking developments and advancements we see in health care and defense; thus affording students everywhere access to fulfilling futures.

For more information about the Alliance for Learning Innovation, please visit https://www.alicoalition.org/.

A Bipartisan Health Agenda to Unite America: Innovative Ideas to Strengthen American Wellbeing

As the COVID-19 pandemic has clearly shown – American health is crucial to the health of our nation. Yet American health is under threat from all angles, from escalating chronic deadly diseases like cancer to rising mental health challenges and the growing overdose epidemic. All of these threats contribute to the United States ranking 31st in life expectancy at birth, one of the lowest in the developed world, despite having the highest health spending per capita.

At the State of the Union, the Biden Administration presented a bipartisan platform dedicated to securing the health and wellbeing of the American people, from our Veterans to our youth. An agenda is a first step – unified action on public health comes next. Evidence-based science policy can bring us closer to a healthier future. Since 2020, policy entrepreneurs have developed innovative implementation-ready policy proposals through the Day One Project (D1P) to tackle some of the biggest societal problems. Here are a few that speak to the current moment:

To combat cancer…

With the median monthly cost of cancer drugs topping $10,000, many families cannot afford the costs of caring for their loved ones. Yet, there are 1,100 FDA-approved off-patent generics that could be used for treating cancer, at a fraction of the cost. Congress should appropriate $100 million into Phase III clinical trials of off-patent generics for treating a variety of cancers. This funding can go towards the National Cancer Institute and be implemented through an open-source pharmaceutical R&D framework through accelerated progress towards accessible and affordable cures.

Environmental hazards are a growing driver of cancers, and disproportionately impact rural and disadvantaged communities. Air pollution has been linked to lung cancer, the most deadly cancer for both men and women in the US. An interagency collaboration led by National Oceanic and Atmospheric Administration and leveraging funds from the Inflation Reduction Act could deploy a network on low-cost, real-time, ground-based sensors in all 300 US cities with a population above 100,000 to track particulate matter rates. Connecting this data to relevant providers in these cities, such as federally-qualified community health centers, could inform physicians of high-risk sites to target early screening interventions. Further, materials composing American homes, from housing materials to pipe materials, and even water running in the faucets, have been identified as possible sources of carcinogens. The Biden Administration should launch the President’s Task Force on Healthy Housing and Water for Cancer Prevention to coordinate research, develop the statistical database, and prepare for regulatory actions.

Finally, innovations in primary care can also catch cancer at earlier stages in disease progression. Yet, many rural and disadvantaged communities lack access to primary care. The NIH’s $23 million investment investigating telehealth for cancer care will develop the best care strategies – but labor-market, technical, financial/regulatory barriers, and data barriers will remain for scaling to the broad population. The Biden Administration and Congress will need to collaborate to unlock barriers to delivering healthcare services directly to the American home, through reforming licensure, expanding broadband access, investing in new mobile healthcare devices, expanding Medicare and Medicaid reimbursements, and ensuring data interoperability.

To strengthen mental health…

Digital mental health technologies have enormous potential to combat the growing mental health crisis, as evidenced by the Administration’s plan on mental health research and development. Yet more work remains to build a national infrastructure for successful implementation of digital mental health services. The vast majority of digital mental health technologies are unregulated, as existing FDA standards fail to cover these emerging technologies because many do not make treatment claims. Congress should authorize Health and Human Services (HHS) to develop standards for digital mental health products to ensure clinical effectiveness, data safety, and mitigate risk. Technologies that meet these standards should then be reimbursable through Medicare and Medicaid, which will require further congressional action. Finally, the Substance Abuse and Mental Health Services Administration (SAMHSA) should create a National Center for Digital Mental Health to maintain a database of approved digital products, provide training to providers, and ensure compliance of developers with national standards.

Knowing that tech platforms can be harmful to the youth’s wellbeing, the Congress and the Administration can take several steps to protect children’s privacy. Congress can expand the technological expertise at the Department of Education (ED) to protect children’s privacy and security in schools as well as appropriate $160 million funding to the Federal Trade Commission (FTC) to expand Children’s Online Protection Privacy Act (COPPA) enforcement and further investigate technology companies extracting children’s data. The Administration can commission a task force to identify ways to protect children’s data through existing legislation such as the Family Educational Rights and Privacy Act and COPPA.

To tackle the opioid crisis…

The opioid crisis is claiming thousands of lives every year, and there is bipartisan consensus on action. The Centers for Medicare and Medicaid (CMS) has sought strategies to prevent opioid use disorders – which will require reforms to the insurance reimbursement model which less generously covers preventative services. The Biden Administration should pilot a multidisciplinary study group to implement payment for prevention, using opioid use disorders as the test case. Following the guidance of the study group, CMS should provide guidelines to contracts between states and managed care organizations (MCOs) and between MCOs and providers and provide necessary technical assistance to implement these guidelines.

To deliver on care for Veterans…

Five million veterans live in rural areas, and of those, 45% lack access to reliable broadband internet, reducing access to vital health services. To ensure Veterans remain connected to healthcare services wherever they are, the Veterans Health Administration (VHA) should partner with the Postal Service and/or Department of Agriculture to pilot telehealth hubs in rural communities using existing FY23 appropriations for telehealth. An initial focus of care delivery could be on digital mental health and suicide prevention. Going forward, care delivery innovations like this one, if successful, can inspire new policies for the broader population, if the VHA’s health policy mission is expanded. VHA should be added to strategic interagency health policy coalitions such as the ACA interagency working group on healthcare quality and Healthy People 2030 to share data, develop innovative projects, and evaluate progress.

There’s more work to be done to build a healthier future for all Americans – these ideas can be jumping off points for Executive and Congressional action. FAS will continue to develop and surface evidence-based policies that can make a difference, and submissions to the Day One project are always welcome.

Tilling the Federal SOIL for Transformative R&D: The Solution Oriented Innovation Liaison

Summary 

The federal government is increasingly embracing Advanced Research Projects Agencies (ARPAs) and other transformative research and engagement enterprises (TREEs) to connect innovators and create the breakthroughs needed to solve complex problems. Our innovation ecosystem needs more of these TREEs, especially for societal challenges that have not historically benefited from solution-oriented research and development. And because the challenges we face are so interwoven, we want them to work and grow together in a solution-oriented mode. 

The National Science Foundation (NSF)’s new Directorate for Technology, Innovation and Partnerships should establish a new Office of the Solution-Oriented Innovation Liaison (SOIL) to help TREEs share knowledge about complementary initiatives, establish a community of practice among breakthrough innovators, and seed a culture for exploring new models of research and development within the federal government. The SOIL would have two primary goals: (1) provide data, information, and knowledge-sharing services across existing TREEs; and (2) explore opportunities to pilot R&D models of the future and embed breakthrough innovation models in underleveraged agencies.

Challenge and Opportunity

Climate change. Food security. Social justice. There is no shortage of complex challenges before us—all intersecting, all demanding civil action, and all waiting for us to share knowledge. Such challenges remain intractable because they are broader than the particular mental models that any one individual or organization holds. To develop solutions, we need science that is more connected to social needs and to other ways of knowing. Our problem is not a deficit of scientific capital. It is a deficit of connection.

Connectivity is what defines a growing number of approaches to the public administration of science and technology, alternatively labeled as transformative innovation, mission-oriented innovation, or solutions R&D. Connectivity is what makes DARPA, IARPA, and ARPA-E work, and it is why new ARPAs are being created for health and proposed for infrastructure, labor, and education. Connectivity is also a common element among an explosion of emerging R&D models, including Focused Research Organizations (FROs) and Distributed Autonomous Organizations (DAOs). And connectivity is the purpose of NSF’s new Directorate for Technology, Innovation and Partnerships (TIP), which includes “fostering innovation ecosystems” in its mission. New transformative research and engagement enterprises (TREEs) could be especially valuable in research domains at the margins, where “the benefits of innovation do not simply trickle down.

The history of ARPAs and other TREEs shows that solutions R&D is successfully conducted by entities that combine both research and engagement. If grown carefully, such organisms bear fruit. So why just plant one here or there when we could grow an entire forest? The metaphor is apt. To grow an innovation ecosystem, we must intentionally sow the seeds of TREEs, nurture their growth, and cultivate symbiotic relationships—all while giving each the space to thrive.

Plan of Action

NSF’s TIP directorate should create a new Office of Solution-Oriented Innovation (SOIL) to foster a thriving community of TREEs. SOIL would have two primary goals: (1) nurture more TREEs of more varieties in more mission spaces; and (2) facilitate more symbiosis among TREEs of increasing number and variety. 

Goal 1: More TREEs of more varieties in more mission spaces

SOIL would shepherd the creation of TREEs wherever they are needed, whether in a federal department, a state or local agency, or in the private, nonprofit, or academic sectors. Key to this is codifying the lessons of successful TREEs and translating them to new contexts. Not all such knowledge is codifiable; much is tacit. As such, SOIL would draw upon a cadre of research-management specialists who have a deep familiarity with different organizational forms (e.g., ARPAs, FROs, DAOs) and could work with the leaders of departments, businesses, universities, consortia, etc. to determine which form best suits the need of the entity in question and provide technical assistance in establishment.

An essential part of this work would be helping institutions create mission-appropriate governance models and cultures. Administering TREEs is neither easy nor typical. Indeed, the very fact that they are managed differently from normal R&D programs makes them special. Former DARPA Director Arati Prabhakar has emphasized the importance of such tailored structures to the success of TREEs. To this end, SOIL would also create a Community of Cultivators comprising former TREE leaders, principal investigators (PIs), and staff. Members of this community would provide those seeking to establish new TREEs with guidance during the scoping, launch, and management phases.

SOIL would also provide opportunities for staff at different TREEs to connect with each other and with collective resources. It could, for example, host dedicated liaison officers at agencies (as DARPA has with its service lines) to coordinate access to SOIL resources and other TREEs and support the documentation of lessons learned for broader use. SOIL could also organize periodic TREE conventions for affiliates to discuss strategic directions and possibly set cross-cutting goals. Similar to the SBIR office at the Small Business Administration, SOIL would also report annually to Congress on the state of the TREE system, as well as make policy recommendations.

Goal 2: More symbiosis among TREEs of increasing number and variety

Success for SOIL would be a community of TREEs that is more than the sum of its parts. It is already clear how the defense and intelligence missions of DARPA and IARPA intersect. There are also energy programs at DARPA that might benefit from deeper engagement with programs at ARPA-E. In the future, transportation-infrastructure programs at ARPA-E could work alongside similar programs at an ARPA for infrastructure. Fostering stronger connections between entities with overlapping missions would minimize redundant efforts and yield shared platform technologies that enable sector-specific advances.

Indeed, symbiotic relationships could spawn untold possibilities. What if researchers across different TREEs could build knowledge together? Exchange findings, data, algorithms, and ideas? Co-create shared models of complex phenomena and put competing models to the test against evidence? Collaborate across projects, and with stakeholders, to develop and apply digital technologies as well as practices to govern their use? A common digital infrastructure and virtual research commons would enable faster, more reliable production (and reproduction) of research across domains. This is the logic underlying the Center for Open Science and the National Secure Data Service.

To this end, SOIL should build a digital Mycelial Network (MyNet), a common virtual space that would harness the cognitive diversity across TREEs for more robust knowledge and tools. MyNet would offer a set of digital services and resources that could be accessed by TREE managers, staff, and PIs. Its most basic function could be to depict the ecosystem of challenges and solutions, search for partners, and deconflict programs. Once partnerships are made, higher-level functions would include secure data sharing, co-creation of solutions, and semantic interconnection. MyNet could replace the current multitude of ad hoc, sector-specific systems for sharing research resources, giving more researchers access to more knowledge about complex systems and fewer obstacles from paywalls. And the larger the network, the bigger the network effects. If the MyNet infrastructure proves successful for TREEs, it could ultimately be expanded more broadly to all research institutions—just as ARPAnet expanded into the public internet. 

For users, MyNet would have three layers:

These functions would collectively require:

How might MyNet be applied? Consider three hypothetical programs, all focused on microplastics: a medical program that maps how microplastics are metabolized and impact health; a food-security program that maps how microplastics flow through food webs and supply chains; and a social justice program that maps which communities produce and consume microplastics. In the data layer, researchers at the three programs could combine data on health records, supply logistics, food inspections, municipal records, and demographics. In the information layer, they might collaborate on coding and evaluating quantitative models. Finally, in the knowledge layer, they could work together to validate claims regarding who is impacted, how much, and by what means.

Initial Steps

First, Congress should authorize and appropriate the NSF TIP Directorate with $500 million over four years for a new Office of the Solution-Oriented Innovation Liaison. Congress should view SOIL as an opportunity to create a shared service among emergent, transformative federal R&D efforts that will empower—rather than bureaucratically stifle—the science and technological advances we need most. This mission fits squarely under the NSF TIP Directorate’s mandate to “mobilize the collective power of the nation” by serving as “a crosscutting platform that collaboratively integrates with NSF’s existing directorates and fosters partnerships—with government, industry, nonprofits, civil society and communities of practice—to leverage, energize and rapidly bring to society use-inspired research and innovation.” 

Once appropriated and authorized to begin intentionally growing a network of TREEs, NSF’s TIP Directorate should focus on a four-year plan for SOIL. TIP should begin by choosing an appropriate leader for SOIL, such as a former director or directorate manager of an ARPA (or other TREE). SOIL would be tasked with first engaging the management of existing ARPAs in the federal government, such as those at the Departments of Defense and Energy, to form an advisory board. The advisory board would in turn guide the creation of experience-informed operating procedures for SOIL to use to establish and aid new TREEs. These might include discussions geared toward arriving at best practices and mechanisms to operate rapid solutions-focused R&D programs for the following functions:

Beyond these structural aspects, the board must also incorporate important cultural aspects of TREES into best practices. In my own research into the managerial heuristics that guide TREEs, I found that managers must be encouraged to “drive change” (critique the status quo, dream big, take action), “be better” (embrace difference, attract excellence, stand out from the crowd), “herd nerds” (focus the creative talent of scientists and engineers), “gather support” (forge relationships with research conductors and potential adversaries), “try and err” (take diverse approaches, expect to fail, learn from failure), and “make it matter” (direct activities to realize outcomes for society, not for science).

The board would also recommend a governance structure and implementation strategy for MyNet. In its first year, SOIL could also start to grow the Community of Cultivators, potentially starting with members of the advisory board. The board chair, in partnership with the White House Office of Science and Technology Policy, would also convene an initial series of interagency working groups (IWGs) focused on establishing a community of practice around TREEs, including but not limited to representatives from the following R&D agencies, offices, and programs: 

In years two and three, SOIL would focus on growing three to five new TREEs at organizations that have not had solutions-oriented innovation programs before but need them. 

SOIL would also start to build a pilot version of MyNet as a resource for these new TREEs, with a goal of including existing ARPAs and other TREEs as quickly as possible. In establishing MyNet, SOIL should focus on implementing the most appropriate system of data governance by first understanding the nature of the collaborative activities intended. Digital research collaborations can apply and mix a range of different governance patterns, with different amounts of availability and freedoms with respect to digital resources. MyNet should be flexible enough to meet a range of needs for openness and security. To this end, SOIL should coordinate with the recently created National Secure Data Service and apply lessons forward in creating an accessible, secure, and ethical information-sharing environment. 

Year four and beyond would be characterized by scaling up. Building on the lessons learned in the prior two years of pilot programs, SOIL would coordinate with new and legacy TREEs to refresh operating procedures and governance structures. It would then work with an even broader set of organizations to increase the number of TREEs beyond the three to five pilots and continue to build out MyNet as well as the Community of Cultivators. Periodic evaluations of SOIL’s programmatic success would shape its evolution after this point. These should be framed in terms of its capacity to create and support programs that yield meaningful technological and socioeconomic outcomes, not just produce traditional research metrics. As such, in its creation of new TREEs, SOIL should apply a major lesson of the National Academies’ evaluation of ARPA-E: explicitly align the (necessarily) robust performance management systems at the project level with strategy and evaluation systems at the program, portfolio, and agency levels. The long-term viability of SOIL and TREEs will depend on their ability to demonstrate value to the public.

Frequently Asked Questions
What is the transformative research model? What makes it different from a typical R&D model?

The transformative research model typically works like this:



  • Engage with stakeholders to understand their needs and set audacious goals for addressing them.

  • Establish lean projects run by teams of diverse experts assembled just long enough to succeed or fail in one approach.

  • Continuously evaluate projects, build on what works, kill what doesn’t, and repeat as necessary.


In a nutshell, transformative research enterprises exist solely to solve a particular problem, rather than to grow a program or amass a stock of scientific capital.


To get more specific, Bonvillian and Van Atta (2011) identify the unique factors that contribute to the innovative nature of ARPAs. On the personnel front, ARPA program managers are talented managers, experienced in business, and appointed for limited terms. They are “translators,” as opposed to subject-matter experts, who actively engage with allies, rivals, and others. They have great power to choose projects, hire, fire, and contract. On the structure front, projects are driven by specific challenges or visions—co-developed with stakeholders—designed around plausible implementation pathways. Projects are executed extramurally, and managed as portfolios, with clear metrics to asses risk and reward. Success for ARPAs means developing products and services that achieve broad uptake and cost-efficacy, so finding first adopters and creating markets is part of the work.

What kinds of TREEs could SOIL help to create?

Some examples come from other Day One proposals. SOIL could work with the Department of Labor to establish a Labor ARPA. It could work with the Department of Education on an Education ARPA. We could imagine a Justice Department ARPA with a program for criminal justice reform, one at Housing and Urban Development aimed at solving homelessness, or one at the State Department for innovations in diplomacy. And there are myriad opportunities beyond the federal government.

What kind of authority over TREEs should SOIL have? Since TREEs are designed to be nimble and independent, wouldn’t SOIL oversight inhibit their operations with an extra layer of bureaucracy?

TREEs thrive on their independence and flexibility, so SOIL’s functions must be designed to impose minimal interference. Other than ensuring that the TREEs it supports are effectively administered as transformative, mission-oriented organizations, SOIL would be very hands-off. SOIL would help establish TREEs and set them up so they do not operate as typical R&D units. SOIL would give TREE projects and staff the means to connect cross-organizationally with other projects and staff in areas of mutual interest (e.g., via MyNet, the Community of Cultivators, and periodic convenings). And, like the SBIR office at the Small Business Administration, SOIL would report annually to Congress on its operations and progress toward goals.

What is the estimated cost of SOIL and its component initiatives? How would it be funded?

An excellent model for SOIL is the Small Business Innovative Research (SBIR) system. SBIR is funded by redirecting a small percentage of the budgets of agencies that spend $100 million or more on extramural R&D. Given that SOIL is intended to be relevant to all federal mission spaces, we recommend that SOIL be funded by a small fraction (between 0.1 and 1.0%) of the budgets of all agencies with $1 billion or more in total discretionary spending. This would yield about $15 billion to support SOIL in growing and connecting new TREEs in a vastly widened set of mission spaces. 


The risk is the opportunity cost of this budget reallocation to each funding agency. It is worth noting, though, that changes of 0.1–1.0% are less than the amount that the average agency sees as annual perturbations in its budget. Moreover, redirecting these funds may well be worth the opportunity cost, especially as an investment in solving the compounding problems that federal agencies face. By redirecting this small fraction of funds, we can keep agency operations 99–99.9% as effective while simultaneously creating a robust, interconnected, solutions-oriented R&D system.

Advanced Research Priorities in Transportation

The Federation of American Scientists (FAS) has identified several domains in the transportation and infrastructure space that retain a plethora of unsolved opportunities ripe for breakthrough innovation.

Transportation is not traditionally viewed as a research- and development-led field, with less than 0.7% of the U.S. Department of Transportation (DOT) annual budget dedicated to R&D activities. The majority of DOT’s R&D funds are disbursed by modal operating administrators mandated to execute on distinct funding priorities rather than a collective, integrated vision of transforming the nation’s infrastructure across 50 states and localities. 

Historically, a small percentage of these R&D funds have supported and developed promising, cross-cutting initiatives, such as the Federal Highway Administration’s Exploratory Advanced Research programs deploying artificial intelligence to better understand driver behavior and applying novel data integration techniques to enhance freight logistics. Yet, the scope of these programs has not been designed to scale discoveries into broad deployment, limiting the impact of innovation and technology in transforming transportation and infrastructure in the United States. 

As a result, transportation and infrastructure retain a plethora of unaddressed opportunities – from reducing the 40,000 annual vehicle-related fatalities, to improving freight logistics through ports, highways, and rail, to achieving a net zero carbon transportation system, to building infrastructure resilient to the impacts of climate change and severe weather. The reasons for these persistent challenges are numerous: low levels of federal R&D spending, fragmentation across state and local government, risk-averse procurement practices, sluggish commercial markets, and more. When innovations do emerge in this field, they suffer from two valleys of death: one to bring new ideas out of the lab into commercialization, and the second to bring successful deployments of those technologies to scale.

The United States needs a concerted national innovation pipeline designed to fill this gap, exploring early-stage, moonshot research while nurturing  breakthroughs from concept to deployment. An Advanced Research Projects Agency-Infrastructure would deliver on this mission. Modeled after the Defense Advanced Research Projects Agency (DARPA) and the Advanced Research Projects Agency-Energy (ARPA-E), the Advanced Research Projects Agency-Infrastructure (ARPA-I) will operate nimbly and with rigorous program management and deep technical expertise to tackle the biggest infrastructure  challenges and overcome entrenched market failures. Solutions would cut across traditional transportation modes (e.g. highways, rail, aviation, maritime, pipelines etc) and would include innovative new infrastructure technologies, materials, systems, capabilities, or processes. 

The list of domain areas below reflects priorities for DOT as well as areas where there is significant opportunity for breakthrough innovation:

Key Domain Areas

Metropolitan Safety

Despite progress made since 1975, dramatic reductions in roadway fatalities remain a core, persistent challenge. In 2021, an estimated 42,915 people were killed in motor vehicle crashes, with an estimated 31,785 people killed in the first nine months of 2022. The magnitude of this challenge is articulated in DOT’s most recent National Roadway Safety Strategy, a document that begins with a statement from Secretary Buttigieg: “The status quo is unacceptable, and it is preventable… Zero is the only acceptable number of deaths and serious injuries on our roadways.” 

Example topical areas include but are not limited to: urban roadway safety; advanced vehicle driver assistance systems; driver alcohol detection systems; vehicle design; street design; speeding and speed limits; and V2X (vehicle-to-everything) communications and networking technology.

Key Questions for Consideration:

Rural Safety

Rural communities possess their own unique safety challenges stemming from road design and signage, speed limits, and other factors; and data from the Federal Highway Administration shows that “while only 19% of the U.S. population lives in rural areas, 43% of all roadway fatalities occur on rural roads, and the fatality rate on rural roads is almost 2 times higher than on urban roads.”

Example topical areas include but are not limited to: improved information collection and management systems; design and evaluation tools for two-lane highways and other geometric design decisions; augmented visibility; mitigating or anti-rollover crash solutions; and enhanced emergency response. 

Key Questions for Consideration:

Resilient & Climate Prepared Infrastructure

Modern roads, bridges, and transportation are designed to withstand storms that, at the time of their construction, had a probability of occurring once in 100 years; today, climate change has made extreme weather events commonplace. In 2020 alone, the U.S. suffered 22 high-impact weather disasters that each cost over $1 billion in damages. When Hurricane Sandy hit New York City and New Jersey subways with a 14-foot storm surge, millions were left without their primary mode of transportation for a week. Meanwhile, rising sea levels are likely to impact both marine and air transportation, as 13 of the 47 largest U.S. airports have at least one runway within 12 feet of the current sea level. Additionally, the persistent presence of wildfires–which are burning an average of 7 million acres annually across the United States, more than double the average in the 1990s–dramatically reshapes the transportation network in acute ways and causes downstream damage through landslides, flooding, and other natural events.

These trends are likely to continue as climate change exacerbates the intensity and scope of these events. The Department of Transportation is well-positioned to introduce systems-level improvements to the resilience of our nation’s infrastructure.

Example topical areas include but are not limited to: High-performance long-life, advanced materials that increase resiliency and reduce maintenance and reconstruction needs, especially materials for roads, rail, and ports; nature-based protective strategies such as constructed marshes; novel designs for multi-modal hubs or other logistics/supply chain redundancy; efficient and dynamic mechanisms to optimize the relocation of transportation assets; intensive maintenance, preservation, prediction, and degradation analysis methods; and intelligent disaster-resilient infrastructure countermeasures. 

Key Questions for Consideration:

Digital Infrastructure

Advancing the systems, tools, and capabilities for digital infrastructure to reflect and manage the built environment has the power to enable improved asset maintenance and operations across all levels of government, at scale. Advancements in this field would make using our infrastructure more seamless for transit, freight, pedestrians, and more. Increased data collection from or about vehicle movements, for example, enables user-friendly and demand-responsive traffic management, dynamic curb management for personal vehicles, transit and delivery transportation modes, congestion pricing, safety mapping and targeted interventions, and rail and port logistics. When data is accessible by local departments of transportation and municipalities, it can be harnessed to improve transportation operations and public safety through crash detection as well as to develop Smart Cities and Communities that utilize user-focused mobility services; connected and automated vehicles; electrification across transportation modes, and intelligent, sensor-based infrastructure to measure and manage age-old problems like potholes, air pollution, traffic, parking, and safety.

Example topical areas include but are not limited to: traffic management; curb management; congestion pricing; accessibility; mapping for safety; rail management; port logistics; and transportation system/electric grid coordination.

Key Questions for Consideration:

Expediting and Upgrading Construction Methods

Infrastructure projects are fraught with expensive delays and overrun budgets. In the United States, fewer than 1 in 3 contractors report finishing projects on time and within budgets, with 70% citing coordination at the site of construction as the primary reason. In the words of one industry executive, “all [of the nation’s] major projects have cost and schedule issues … the truth is these are very high-risk and difficult projects. Conditions change. It is impossible to estimate it accurately.” But can process improvements and other innovations make construction cheaper, better, faster, and easier?

Example topical areas include but are not limited to: augmented forecasting and modeling techniques; prefabricated or advanced robotic fabrication, modular, and adaptable structures and systems such as bridge sub- and superstructures; real-time quality control and assurance technologies for accelerated construction, materials innovation; new pavement technologies; bioretention; tunneling; underground infrastructure mapping; novel methods for bridge engineering, building information modeling (BIM), coastal, wind, and offshore engineering; stormwater systems; and computational methods in structural engineering, structural sensing, control, and asset management. 

Key Questions for Consideration:

Logistics

Our national economic strength and quality of life depend on the safe and efficient movement of goods throughout our nation’s borders and beyond. Logistic systems—the interconnected webs of businesses, workers, infrastructure processes, and practices that underlie the sorting, transportation, and distribution of goods must operate with efficiency and resilience. . When logistics systems are disrupted by events such as public health crises, extreme weather, workforce challenges, or cyberattacks, goods are delayed, costs increase, and Americans’ daily lives are affected. The Biden Administration issued Executive Order 14017 calling for a review of the transportation and logistics industrial base. DOT released the Freight and Logistics Supply Chain Assessment in February 2022, spotlighting a range of actions that DOT envisions to support a resilient 21st-century freight and logistics supply chain for America.

Topical areas include but are not limited to: freight infrastructure, including ports, roads, airports, and railroads; data and research; rules and regulations; coordination across public and private sectors; and supply chain electrification and intersections with resilient infrastructure. 

Key Questions for Consideration: