Ecosystems & Entrepreneurship
day one project

Tilling the Federal SOIL for Transformative R&D: The Solution Oriented Innovation Liaison

01.10.23 | 11 min read | Text by Avery Sen


The federal government is increasingly embracing Advanced Research Projects Agencies (ARPAs) and other transformative research and engagement enterprises (TREEs) to connect innovators and create the breakthroughs needed to solve complex problems. Our innovation ecosystem needs more of these TREEs, especially for societal challenges that have not historically benefited from solution-oriented research and development. And because the challenges we face are so interwoven, we want them to work and grow together in a solution-oriented mode. 

The National Science Foundation (NSF)’s new Directorate for Technology, Innovation and Partnerships should establish a new Office of the Solution-Oriented Innovation Liaison (SOIL) to help TREEs share knowledge about complementary initiatives, establish a community of practice among breakthrough innovators, and seed a culture for exploring new models of research and development within the federal government. The SOIL would have two primary goals: (1) provide data, information, and knowledge-sharing services across existing TREEs; and (2) explore opportunities to pilot R&D models of the future and embed breakthrough innovation models in underleveraged agencies.

Challenge and Opportunity

Climate change. Food security. Social justice. There is no shortage of complex challenges before us—all intersecting, all demanding civil action, and all waiting for us to share knowledge. Such challenges remain intractable because they are broader than the particular mental models that any one individual or organization holds. To develop solutions, we need science that is more connected to social needs and to other ways of knowing. Our problem is not a deficit of scientific capital. It is a deficit of connection.

Connectivity is what defines a growing number of approaches to the public administration of science and technology, alternatively labeled as transformative innovation, mission-oriented innovation, or solutions R&D. Connectivity is what makes DARPA, IARPA, and ARPA-E work, and it is why new ARPAs are being created for health and proposed for infrastructure, labor, and education. Connectivity is also a common element among an explosion of emerging R&D models, including Focused Research Organizations (FROs) and Distributed Autonomous Organizations (DAOs). And connectivity is the purpose of NSF’s new Directorate for Technology, Innovation and Partnerships (TIP), which includes “fostering innovation ecosystems” in its mission. New transformative research and engagement enterprises (TREEs) could be especially valuable in research domains at the margins, where “the benefits of innovation do not simply trickle down.

The history of ARPAs and other TREEs shows that solutions R&D is successfully conducted by entities that combine both research and engagement. If grown carefully, such organisms bear fruit. So why just plant one here or there when we could grow an entire forest? The metaphor is apt. To grow an innovation ecosystem, we must intentionally sow the seeds of TREEs, nurture their growth, and cultivate symbiotic relationships—all while giving each the space to thrive.

Plan of Action

NSF’s TIP directorate should create a new Office of Solution-Oriented Innovation (SOIL) to foster a thriving community of TREEs. SOIL would have two primary goals: (1) nurture more TREEs of more varieties in more mission spaces; and (2) facilitate more symbiosis among TREEs of increasing number and variety. 

Goal 1: More TREEs of more varieties in more mission spaces

SOIL would shepherd the creation of TREEs wherever they are needed, whether in a federal department, a state or local agency, or in the private, nonprofit, or academic sectors. Key to this is codifying the lessons of successful TREEs and translating them to new contexts. Not all such knowledge is codifiable; much is tacit. As such, SOIL would draw upon a cadre of research-management specialists who have a deep familiarity with different organizational forms (e.g., ARPAs, FROs, DAOs) and could work with the leaders of departments, businesses, universities, consortia, etc. to determine which form best suits the need of the entity in question and provide technical assistance in establishment.

An essential part of this work would be helping institutions create mission-appropriate governance models and cultures. Administering TREEs is neither easy nor typical. Indeed, the very fact that they are managed differently from normal R&D programs makes them special. Former DARPA Director Arati Prabhakar has emphasized the importance of such tailored structures to the success of TREEs. To this end, SOIL would also create a Community of Cultivators comprising former TREE leaders, principal investigators (PIs), and staff. Members of this community would provide those seeking to establish new TREEs with guidance during the scoping, launch, and management phases.

SOIL would also provide opportunities for staff at different TREEs to connect with each other and with collective resources. It could, for example, host dedicated liaison officers at agencies (as DARPA has with its service lines) to coordinate access to SOIL resources and other TREEs and support the documentation of lessons learned for broader use. SOIL could also organize periodic TREE conventions for affiliates to discuss strategic directions and possibly set cross-cutting goals. Similar to the SBIR office at the Small Business Administration, SOIL would also report annually to Congress on the state of the TREE system, as well as make policy recommendations.

Goal 2: More symbiosis among TREEs of increasing number and variety

Success for SOIL would be a community of TREEs that is more than the sum of its parts. It is already clear how the defense and intelligence missions of DARPA and IARPA intersect. There are also energy programs at DARPA that might benefit from deeper engagement with programs at ARPA-E. In the future, transportation-infrastructure programs at ARPA-E could work alongside similar programs at an ARPA for infrastructure. Fostering stronger connections between entities with overlapping missions would minimize redundant efforts and yield shared platform technologies that enable sector-specific advances.

Indeed, symbiotic relationships could spawn untold possibilities. What if researchers across different TREEs could build knowledge together? Exchange findings, data, algorithms, and ideas? Co-create shared models of complex phenomena and put competing models to the test against evidence? Collaborate across projects, and with stakeholders, to develop and apply digital technologies as well as practices to govern their use? A common digital infrastructure and virtual research commons would enable faster, more reliable production (and reproduction) of research across domains. This is the logic underlying the Center for Open Science and the National Secure Data Service.

To this end, SOIL should build a digital Mycelial Network (MyNet), a common virtual space that would harness the cognitive diversity across TREEs for more robust knowledge and tools. MyNet would offer a set of digital services and resources that could be accessed by TREE managers, staff, and PIs. Its most basic function could be to depict the ecosystem of challenges and solutions, search for partners, and deconflict programs. Once partnerships are made, higher-level functions would include secure data sharing, co-creation of solutions, and semantic interconnection. MyNet could replace the current multitude of ad hoc, sector-specific systems for sharing research resources, giving more researchers access to more knowledge about complex systems and fewer obstacles from paywalls. And the larger the network, the bigger the network effects. If the MyNet infrastructure proves successful for TREEs, it could ultimately be expanded more broadly to all research institutions—just as ARPAnet expanded into the public internet. 

For users, MyNet would have three layers:

These functions would collectively require:

How might MyNet be applied? Consider three hypothetical programs, all focused on microplastics: a medical program that maps how microplastics are metabolized and impact health; a food-security program that maps how microplastics flow through food webs and supply chains; and a social justice program that maps which communities produce and consume microplastics. In the data layer, researchers at the three programs could combine data on health records, supply logistics, food inspections, municipal records, and demographics. In the information layer, they might collaborate on coding and evaluating quantitative models. Finally, in the knowledge layer, they could work together to validate claims regarding who is impacted, how much, and by what means.

Initial Steps

First, Congress should authorize and appropriate the NSF TIP Directorate with $500 million over four years for a new Office of the Solution-Oriented Innovation Liaison. Congress should view SOIL as an opportunity to create a shared service among emergent, transformative federal R&D efforts that will empower—rather than bureaucratically stifle—the science and technological advances we need most. This mission fits squarely under the NSF TIP Directorate’s mandate to “mobilize the collective power of the nation” by serving as “a crosscutting platform that collaboratively integrates with NSF’s existing directorates and fosters partnerships—with government, industry, nonprofits, civil society and communities of practice—to leverage, energize and rapidly bring to society use-inspired research and innovation.” 

Once appropriated and authorized to begin intentionally growing a network of TREEs, NSF’s TIP Directorate should focus on a four-year plan for SOIL. TIP should begin by choosing an appropriate leader for SOIL, such as a former director or directorate manager of an ARPA (or other TREE). SOIL would be tasked with first engaging the management of existing ARPAs in the federal government, such as those at the Departments of Defense and Energy, to form an advisory board. The advisory board would in turn guide the creation of experience-informed operating procedures for SOIL to use to establish and aid new TREEs. These might include discussions geared toward arriving at best practices and mechanisms to operate rapid solutions-focused R&D programs for the following functions:

Beyond these structural aspects, the board must also incorporate important cultural aspects of TREES into best practices. In my own research into the managerial heuristics that guide TREEs, I found that managers must be encouraged to “drive change” (critique the status quo, dream big, take action), “be better” (embrace difference, attract excellence, stand out from the crowd), “herd nerds” (focus the creative talent of scientists and engineers), “gather support” (forge relationships with research conductors and potential adversaries), “try and err” (take diverse approaches, expect to fail, learn from failure), and “make it matter” (direct activities to realize outcomes for society, not for science).

The board would also recommend a governance structure and implementation strategy for MyNet. In its first year, SOIL could also start to grow the Community of Cultivators, potentially starting with members of the advisory board. The board chair, in partnership with the White House Office of Science and Technology Policy, would also convene an initial series of interagency working groups (IWGs) focused on establishing a community of practice around TREEs, including but not limited to representatives from the following R&D agencies, offices, and programs: 

In years two and three, SOIL would focus on growing three to five new TREEs at organizations that have not had solutions-oriented innovation programs before but need them. 

SOIL would also start to build a pilot version of MyNet as a resource for these new TREEs, with a goal of including existing ARPAs and other TREEs as quickly as possible. In establishing MyNet, SOIL should focus on implementing the most appropriate system of data governance by first understanding the nature of the collaborative activities intended. Digital research collaborations can apply and mix a range of different governance patterns, with different amounts of availability and freedoms with respect to digital resources. MyNet should be flexible enough to meet a range of needs for openness and security. To this end, SOIL should coordinate with the recently created National Secure Data Service and apply lessons forward in creating an accessible, secure, and ethical information-sharing environment. 

Year four and beyond would be characterized by scaling up. Building on the lessons learned in the prior two years of pilot programs, SOIL would coordinate with new and legacy TREEs to refresh operating procedures and governance structures. It would then work with an even broader set of organizations to increase the number of TREEs beyond the three to five pilots and continue to build out MyNet as well as the Community of Cultivators. Periodic evaluations of SOIL’s programmatic success would shape its evolution after this point. These should be framed in terms of its capacity to create and support programs that yield meaningful technological and socioeconomic outcomes, not just produce traditional research metrics. As such, in its creation of new TREEs, SOIL should apply a major lesson of the National Academies’ evaluation of ARPA-E: explicitly align the (necessarily) robust performance management systems at the project level with strategy and evaluation systems at the program, portfolio, and agency levels. The long-term viability of SOIL and TREEs will depend on their ability to demonstrate value to the public.

Frequently Asked Questions
What is the transformative research model? What makes it different from a typical R&D model?

The transformative research model typically works like this:

  • Engage with stakeholders to understand their needs and set audacious goals for addressing them.

  • Establish lean projects run by teams of diverse experts assembled just long enough to succeed or fail in one approach.

  • Continuously evaluate projects, build on what works, kill what doesn’t, and repeat as necessary.

In a nutshell, transformative research enterprises exist solely to solve a particular problem, rather than to grow a program or amass a stock of scientific capital.

To get more specific, Bonvillian and Van Atta (2011) identify the unique factors that contribute to the innovative nature of ARPAs. On the personnel front, ARPA program managers are talented managers, experienced in business, and appointed for limited terms. They are “translators,” as opposed to subject-matter experts, who actively engage with allies, rivals, and others. They have great power to choose projects, hire, fire, and contract. On the structure front, projects are driven by specific challenges or visions—co-developed with stakeholders—designed around plausible implementation pathways. Projects are executed extramurally, and managed as portfolios, with clear metrics to asses risk and reward. Success for ARPAs means developing products and services that achieve broad uptake and cost-efficacy, so finding first adopters and creating markets is part of the work.

What kinds of TREEs could SOIL help to create?

Some examples come from other Day One proposals. SOIL could work with the Department of Labor to establish a Labor ARPA. It could work with the Department of Education on an Education ARPA. We could imagine a Justice Department ARPA with a program for criminal justice reform, one at Housing and Urban Development aimed at solving homelessness, or one at the State Department for innovations in diplomacy. And there are myriad opportunities beyond the federal government.

What kind of authority over TREEs should SOIL have? Since TREEs are designed to be nimble and independent, wouldn’t SOIL oversight inhibit their operations with an extra layer of bureaucracy?

TREEs thrive on their independence and flexibility, so SOIL’s functions must be designed to impose minimal interference. Other than ensuring that the TREEs it supports are effectively administered as transformative, mission-oriented organizations, SOIL would be very hands-off. SOIL would help establish TREEs and set them up so they do not operate as typical R&D units. SOIL would give TREE projects and staff the means to connect cross-organizationally with other projects and staff in areas of mutual interest (e.g., via MyNet, the Community of Cultivators, and periodic convenings). And, like the SBIR office at the Small Business Administration, SOIL would report annually to Congress on its operations and progress toward goals.

What is the estimated cost of SOIL and its component initiatives? How would it be funded?

An excellent model for SOIL is the Small Business Innovative Research (SBIR) system. SBIR is funded by redirecting a small percentage of the budgets of agencies that spend $100 million or more on extramural R&D. Given that SOIL is intended to be relevant to all federal mission spaces, we recommend that SOIL be funded by a small fraction (between 0.1 and 1.0%) of the budgets of all agencies with $1 billion or more in total discretionary spending. This would yield about $15 billion to support SOIL in growing and connecting new TREEs in a vastly widened set of mission spaces. 

The risk is the opportunity cost of this budget reallocation to each funding agency. It is worth noting, though, that changes of 0.1–1.0% are less than the amount that the average agency sees as annual perturbations in its budget. Moreover, redirecting these funds may well be worth the opportunity cost, especially as an investment in solving the compounding problems that federal agencies face. By redirecting this small fraction of funds, we can keep agency operations 99–99.9% as effective while simultaneously creating a robust, interconnected, solutions-oriented R&D system.