There Are Better Ways to Streamline Satellite Licensing
The Federation of American Scientists supports Congress’ ongoing bipartisan efforts to strengthen U.S. leadership with respect to outer space activities, including the Senate’s unanimous passage of the Orbital Sustainability (ORBITS) Act in 2023, and the introduction of the Dark and Quiet Skies Act in 2024 and the Space Exploration Research Act in 2025.
The Satellite and Telecommunications Streamlining Act (S. 3639) aims to address an important issue regarding the streamlining of approvals at the Federal Communications Commission (FCC) for satellite radiofrequency licensing, including expedited reviews for minor modifications and other special cases. However, we are concerned by the “deemed granted” language in S. 3639, particularly with respect to America’s ability to fulfill the authorization and continuing supervision requirements under Article VI of the Outer Space Treaty. The “deemed granted” provision would automatically approve a license if the review period passes without a decision, effectively allowing the FCC to grant radiofrequency licenses without taking action.
The nature of the outer space environment means that space-based systems are capable of causing significant issues with respect to terrestrial activities, from increased cellular interference to degraded weather forecasting capability and impacts to astronomical research. We believe that more responsible measures, like ensuring adequate staffing and performance management at the FCC would achieve a similar objective. We have recently proposed a slate of recommendations in “ASTRA: An American Space Transformation Regulatory Act”, which include consolidating multiple approval processes under a single space regulatory and promotion agency. It also recommended preserving the FCC’s regulatory authority over spectrum issues “(g)iven the significant terrestrial impact of spectrum issues related to space.” We maintain that a streamlined mission authorization process should be the highest priority for commercial space regulatory simplification.
Licensing processes that do not provide sufficient oversight and have significant terrestrial impact are likely to increase the willingness of international partners to deploy additional extraterritorial regulatory requirements, as has happened in other high-tech sectors. Such regulatory regimes have a tendency to dramatically increase the burden on U.S. companies by introducing duplicative or conflicting licensing requirements, making it more difficult for American companies to compete in international markets. As the commercial space sector depends on international market access for profitability, such an outcome must be avoided.
More broadly, the Federation of American Scientists urges more concerted efforts to establish a clear and simplified pathway for space mission authorization and supervision, consistent with the Outer Space Treaty, that maintains the reputation of the United States government and American companies as responsible actors in the outer space environment.
Position on Regional Leadership in Wildland Fire Research Act
FAS supports the bipartisan Regional Leadership in Wildland Fire Research Act under review in the House, just as we supported the earlier Senate version. Rep. David Min (D-CA-47) and Rep. Gabe Evans (R-CO-08) are leading the bill.
The Regional Leadership in Wildland Fire Research Act would establish regional research centers at institutions of higher education across the country to research and improve our understanding of wildland fire, develop, maintain, and operate next-generation fire and vegetation models, and create a career pathway training program.
“Extreme weather has pushed wildfires to grow in size and severity, making our current wildfire models inadequate,” says Daniel Correa, Chief Executive Officer of the Federation of American Scientists. “The Regional Leadership in Wildland Fire Research Act is a significant investment in understanding how wildland fire risks continue to evolve. It establishes a strong foundation that first responders and forest managers can rely on. We commend Congressmen David Min (D-CA-47) and Gabe Evans (R-CO-08) for their leadership investing in innovative next-generation fire and vegetation models to protect human health, ecosystems, and our communities.”
For more information contact Gil Ruiz or Jessica Blackband.
Re: Request for Information on Technology Roadmap To Increase Wildfire Firefighting Capabilities
Re: Request for Information on Technology Roadmap To Increase Wildfire Firefighting Capabilities (Federal Register Number 2025-18121)
The Federation of American Scientists (FAS) is a nonprofit, nonpartisan organization that works to advance science and technology in the public interest. FAS has established itself as an influential convener, network builder, and thought leader on wildfire issues.
Thank you for the opportunity to provide input to inform the Technology Roadmap To Increase Wildfire Firefighting Capabilities. We will focus this comment on the following elements that we think should be emphasized in the development of the roadmap:
- Leveraging the existing body of work on wildfire technology needs and opportunities;
- Balancing investments in technology for suppression with investments in technology for risk reduction;
- Innovating to protect wildland firefighters from the health impacts of wildfire smoke;
- Providing substantive opportunities for end users of technology to give iterative input;
- Ensuring that employees and first responders can make the best use of available data, including federal data, by prioritizing data standardization and interoperability;
- Addressing administrative barriers to technology adoption.
More information about each of these elements is provided below.
Leveraging the existing body of work on wildfire technology needs and opportunities
The current wildfire management system is inadequate in the face of increasingly severe and damaging wildfires. Change is urgently needed. Acknowledging this need was the impetus behind the consensus-building work of the Wildland Fire Mitigation and Management Commission as well as substantial legislative proposals that have emerged since, including the S. 1462, the Fix Our Forests Act (which FAS endorsed). In developing the roadmap, we encourage you to review the substantial body of work developed in the last several years that provides evidence-based recommendations for improving the use of technology in building wildfire resilience at various levels of government. The roadmap should clearly articulate how it will build on prior federal efforts. Examples of prior work include:
- ON FIRE: The Report of the Wildland Fire Mitigation and Management Commission (2023). Chapter 6, which focuses on integrating modern science and technology, is particularly relevant and includes recommendations related to supporting decision-making and accelerating technology.
- Wildland Fire Mitigation and Management Commission Aerial Equipment Strategy Report (2023). Among other things, this report highlights “the need to develop an overarching, forward-looking aviation strategy that drives procurement…the need to invest in both technology and people to build an aviation fleet that meets long-term demand; and the need to take an inclusive approach to the range of functions aerial resources can serve and the range of entities that must be included in development of a truly national–rather than federal—aviation strategy” and discusses specific mechanisms for achieving these ends.
- President’s Council of Advisors on Science and Technology Report to the President on Modernizing Wildland Firefighting to Protect Our Firefighters (2023). This report provided short- and long-term recommendations for investments to ensure that firefighters are prepared for the future of wildfires.
Balancing investments in technology for suppression with investments in technology for risk reduction
Wildfire suppression is critical for protecting people, property, and livelihoods. But with wildfire suppression costs ballooning, suppression must be efficient, targeted, and used only where needed. Federal agencies should prioritize deploying suppression resources where fire presents an imminent danger to people, property, and livelihoods, such as ignitions in the wildland urban interface (WUI). Thankfully, many of the tools needed for smart suppression (e.g., AI-powered ignition detection, satellite-enabled wildfire monitoring) already exist today. The key is to scale and deploy these tools effectively. Support for research, predictive modeling, and real-time fire data and information is also needed to enable precise, timely wildfire response and suppression.
At the same time, we must recognize that suppression is expensive and can even be counterproductive—over-suppression today can make fires worse tomorrow. Letting wildfires burn in a responsible and controlled way where those fires present limited risk, such as deep within undeveloped forested area, is a common-sense use of resources and will build long-term wildfire resilience.
Therefore, the roadmap should provide a clear vision for how the federal workforce as well as subnational entities will be well-equipped to continue critical risk reduction work in addition to wildfire response. Federal agencies have contributed to a buildup of hazardous wildfire fuels on our landscape by prioritizing suppression for nearly a century—creating risks of catastrophic megafires that even the best tools struggle to suppress. To address this systemic problem, we must invest in preparedness and resilience alongside suppression. Proactive risk management is fiscally responsible; there is a seven-dollar return on investment for every dollar spent on fuel treatments and forest management informed by fire research. Investments in technology can support safe, more targeted prescribed burns and enable prioritization of risk reduction activities that will maximize benefits to ecosystems, recreation areas, and water supplies in addition to reducing the risk of catastrophic wildfire.
Innovating to protect wildland firefighters from health impacts of wildfire smoke
Exposure to PM2.5, a major component of wildfire smoke, has been associated with short and long term health impacts on wildland firefighters. We must do more to protect our wildland firefighters from these health impacts. Specifically, the federal government should develop or procure personal protective equipment (PPE) that is specifically designed for the unique hazards and demands of wildland firefighting. In addition to ensuring sufficient PPE, the federal government should work to better understand the risks posed by long-term exposure to wildfire smoke, communicate those risks, and protect those working alongside wildfire in the built and natural environments. Additional actions that the federal government should take to protect wildland firefighters, discussed in more detail in a recent FAS publication, include studying the long-term impacts of wildland firefighting on human health as well as developing technologies, tactics, and practices that reduce the amount of time that wildland firefighters spend at the highest levels of wildfire smoke exposure.
Providing substantive opportunities for end users of technology to give iterative input
Creating opportunities for land managers and firefighters to provide iterative feedback on the development of tools and technologies can ensure that tools and technologies developed at the federal level are as useful as possible for firefighters, land managers, and others at state and local levels, resulting in a more functional and cost-effective firefighting ecosystem. For digital products, shifting towards a product model rather than project management model can support iterative user input. Jennifer Pahlka, a Senior Fellow with FAS, has written extensively about this product model (which she calls “the art of deciding what to do”) and how to deploy it in government. Applying product management funding mechanisms and approaches in government can result in more functional digital products delivered more quickly. Resourcing, including trained user experience experts, are key to successfully deploying the product management model.
Ensuring that employees and first responders can make the best use of available data, including federal data, by prioritizing data standardization and interoperability
We concur with OSTP that “establishing data standardization and interoperability requirements to facilitate seamless data-sharing, and tools that will improve situational awareness for Federal, State, local, tribal, and territorial governments and private stakeholders” should be a high priority for the federal government. Close coordination with other emerging federal efforts to this end, including activities undertaken by the Wildland Fire Intelligence Center proposed in S. 1462, should be a priority to prevent duplication of effort at various levels of government. Establishing data standards and interoperability requirements will also support more effective development and deployment of artificial intelligence tools for suppression, risk reduction, and recovery.
Coordinated data systems, improved data interoperability, and enhanced data accessibility can help state and local decision-makers make the most of existing information and facilitate development of effective technology. Data managers should seek to integrate state and local data and should be accessible to non-federal partners. When working with Tribes, care should be taken to ensure data sovereignty and confidentiality where requested.
Addressing administrative barriers to technology adoption
Per the Wildland Fire Mitigation and Management Commission’s recommendation 117, agencies should work with Congress to ensure that “procurement and contracting…allow for more flexible partnerships with private industry and non-governmental partners.” See recommendation 117 of the Commission report for more information about specific mechanisms. As one example, Congress could provide relevant federal fire agencies with Other Transaction Authority (OTA) already granted to the National Aeronautic and Space Administration (NASA). Agencies with OTA authority can enter into transactions other than procurement contracts, grants, or cooperative agreements, meaning they do not need to comply with the Federal Acquisition Regulation (FAR). As FAS staff have discussed in prior publications, OTA authorities must be leveraged “to its full statutory extent” by relevant agencies in order to have maximum benefits for technology acquisition.,
Additionally, as the Environmental Policy Innovation Center noted in a 2024 report on adopting innovation in the U.S. Forest Service, it is also crucial for federal government entities to develop “well-known entry points.” Having a designated point of contact at the relevant agency and a clear path to working with the government will support technology providers in bringing their innovations to the public sector. It can also support state and local governments in contacting relevant stakeholders in the federal government to coordinate on technology projects.
FAS Position on “Schedule PC” and Impact on Federal Scientists
FAS shares the following formal comment in the Federal Register and asks that the scientific community, and the people across the nation who benefit from their research, to do the same.
The Federation of American Scientists opposes the proposed “Schedule Policy/Career” (“Schedule PC”) in present form because it rescinds civil servant employment protections, placing unnecessary and undesirable political pressure on highly specialized scientific and technical career professionals serving in government.
FAS encourages the Office of Personnel Management to rescind or substantially overhaul the Proposed Rule on Improving Performance, Accountability and Responsiveness in the Civil Service. We ask that OPM respond to the following comments and reflect how it will revise the Proposed Rule or abandon it.
New Employment Category is Unnecessary
Instead of creating a new employment category – the Schedule P/C for federal civil servants – the same goals can be accomplished by requiring agencies to regularly review and update critical elements in the performance appraisal system and their rating factors. Changing performance elements will have the impact of ensuring attention to accountability and responsiveness to policy without the ambiguity or determining assignment to the Schedule or the taxpayer expense of defending it.
The Administration is already taking this action by changing the performance appraisal system for the Senior Executive Service to make senior executives more responsive to Executive-branch priorities and policies. FAS advocates for updates to performance standards and rating factors appropriate for non-executives–based on the best available evidence–to achieve the intended accountability and responsiveness goals in this Proposed Rule.
Proposed Rule Conflates Accountability with Administration
The Proposed Rule makes several errors in interpretation of the Civil Service Act of 1978, including the one potentially most detrimental to scientific enquiry, innovation, and exploration:
- The proposed rule is about accountability to the President and his/her Administration policies, not about performance on the job and accountability to the Constitution. By conflating the two, Schedule P/C takes away individual appeal rights for anyone reassigned to this categorization rather than focusing on removing poor performers. An employee’s poor performance is more commonly related to a lack of quality, accuracy, and/or timeliness of their job tasks, according to the U.S. Merit Systems Protection Board. As written, Schedule P/C also discourages dissent, evidence-based policymaking, performance management to understand and track results, and program evaluation to understand outcomes.
- The proposed rule newly defines Policy-Influencing Roles for merit-based civil servants, while underutilizing existing regulations for other Policy-Making roles like political appointees and those with excepted service employment.
- Newly designating “Policy-influencing” positions as Schedule P/C provides such a breadth of interpretation for federal agencies that it could encompass most federal jobs, which currently rely on a non-partisan, merit based civil service and their associated civil service protections. Already, a Social Security Administration (SSA) leader has voiced the intent to designate nearly all SSA career employees as Schedule P/C. Furthermore, the lack of guidance to agencies in identifying “policy influencing” roles will create inconsistencies in its application across agencies and confusion in comparing similar occupations and their duties.
- Moreover, the Proposed Rule deviates from the accepted definitions for “policy determining,” “policy advocating,” and “policy influencing” roles identified in the Civil Service Act of 1978, and assigned to political appointees and excepted service employment categories. If the proposed rule were limited to “policy determining” and “policy making”, most of these positions would already be part of the Senior Executive Service (SES). These federal employment Schedules already carry the requisite responsiveness and accountability to Administration policies and priorities needed to ensure alignment of federal programs with legislative and executive branch intent.
- Newly designating “Policy-influencing” positions as Schedule P/C provides such a breadth of interpretation for federal agencies that it could encompass most federal jobs, which currently rely on a non-partisan, merit based civil service and their associated civil service protections. Already, a Social Security Administration (SSA) leader has voiced the intent to designate nearly all SSA career employees as Schedule P/C. Furthermore, the lack of guidance to agencies in identifying “policy influencing” roles will create inconsistencies in its application across agencies and confusion in comparing similar occupations and their duties.
Securing American AI Leadership: A Strategic Action Plan for Innovation, Adoption, and Trust
The Federation of American Scientists (FAS) submitted the following response to the Request for Information (RFI) issued by the Office of Science and Technology Policy (OSTP) in February 2025 regarding the development of an Artificial Intelligence (AI) Action Plan.
At a time when AI is poised to transform every sector of the economy, the Trump administration has a critical opportunity to solidify America’s leadership in this pivotal technology. Building on the foundations laid during the first Trump administration, bold and targeted policies can unleash innovation, unlocking AI’s vast potential to stimulate economic growth, revolutionize industries, and strengthen national security. However, innovation alone is insufficient; without public trust, AI adoption will stall. Ensuring AI systems are transparent, reliable, and aligned with American values will accelerate responsible adoption and solidify AI as a cornerstone of America’s economic and technological leadership.
To sustain America’s leadership in AI innovation, accelerate adoption across the economy, and guarantee that AI systems remain secure and trustworthy, we offer a set of actionable policy recommendations. Developed by FAS in partnership with prominent AI experts, industry leaders, and research institutions—including contributors to the recent FAS Day One 2025 Project and the 2024 AI Legislative Sprint—these proposals are structured around four strategic pillars: 1) unleashing AI innovation, 2) accelerating AI adoption, 3) ensuring secure and trustworthy AI, and 4) strengthening existing world-class U.S. government institutions and programs.
1) Unleashing AI Innovation. American AI leadership has been driven by bold private-sector investments and world-class academic research. However, critical high-impact areas remain underfunded. The federal government can catalyze investment and innovation by expanding access to essential data, investing strategically in overlooked areas of AI R&D, defining priority research challenges, promoting public-private partnerships, and attracting and retaining global talent.
2) Accelerating AI Adoption Across the Economy. The United States leads in AI breakthroughs, but these breakthroughs must translate into widespread adoption to maximize their economic and societal benefits. Accelerating adoption—a critical yet often overlooked driver of national competitiveness—requires addressing workforce readiness, expanding government capacity, and managing rising energy demands.
3) Ensuring Secure and Trustworthy AI. Ensuring AI systems are secure and trustworthy is essential not only for fostering public confidence and accelerating widespread adoption, but also for improving government efficiency and ensuring the responsible use of taxpayer resources when AI is deployed by public agencies. While the previous Trump administration recognized the necessity of public trust when promoting AI adoption, concerns persist about AI’s rapid evolution, unpredictable capabilities, and potential for misuse. Future AI accidents could further erode this trust, stalling AI progress. To address these risks and fully harness AI’s potential, the U.S. government must proactively monitor emerging threats, rigorously evaluate AI technologies, and encourage innovation that upholds fundamental American values such as privacy.
4) Strengthening Existing World-Class U.S. Government AI Institutions and Programs. Realizing the Trump Administration’s goals will require building on leading government AI capabilities. Key initiatives—including the NIST AI Safety Institute (AISI), the National AI Research Resource (NAIRR) Pilot, the AI Use Case Inventory, and the Department of Energy’s Office of Critical and Emerging Technologies (CET)—advance AI innovation, security, and transparency. The AISI evaluates AI models with broad industry support, while the NAIRR Pilot expands access to AI resources beyond Big Tech. Federal AI use case inventories enhance government transparency and industry engagement, building public trust. DOE’s CET drives AI-powered advancements in science and national security. Integrating these proven initiatives into the AI Action Plan will solidify America’s AI leadership.
By acting decisively, the administration can ensure American AI remains the gold standard, drive economic competitiveness, and accelerate science and innovation.
Overview of Policy Proposals
Policy Proposals to Unleash AI Innovation
- Recommendation 1: Promote innovation in trustworthy AI through a Public-Private National Initiative for AI Explainability.
- Recommendation 2: Direct the Department of Energy (DOE) to use AI to accelerate the discovery of new materials.
- Recommendation 3: Create AI-ready collaborative datasets to accelerate progress in the life sciences.
- Recommendation 4: Establish a NIST Foundation to amplify public-private collaboration, secure private investment, and accelerate innovation.
- Recommendation 5: Attract top global talent by creating a National Security AI Entrepreneur Visa for elite dual-use technology founders.
Policy Proposals to Accelerate AI Adoption Across the Economy
- Recommendation 1: Streamline procurement processes for government use of AI.
- Recommendation 2: Establish a Federal Center of Excellence to expand state and local government capacity for AI procurement and use.
- Recommendation 3: Pilot an AI Corps at HHS to drive government-wide AI adoption.
- Recommendation 4: Make America’s teacher workforce competitive for the AI era.
- Recommendation 5: Prepare U.S. energy infrastructure for AI growth through standardized measurement and forecasting.
Policy Proposals to Ensure Secure and Trustworthy AI
- Privacy:
- Recommendation 1: Secure third party commercial data for AI through FedRAMP authorization.
- Recommendation 2: Catalyze federal data sharing through privacy enhancing technologies.
- Recommendation 3: Establish data-sharing standards to support AI development in healthcare.
- Security, Safety, and Trustworthiness:
- Recommendation 1: Establish an early warning system for AI-powered threats to national security and public safety.
- Recommendation 2: Create a voluntary AI incident reporting hub to monitor security incidents from AI.
- Recommendation 3: Promote AI trustworthiness by providing a safe harbor for AI researchers.
- Recommendation 4: Build a national digital content authentication technologies research ecosystem.
- Recommendation 5: Strengthen national security by evaluating AI-driven biological threats.
Policy Proposals to Strengthen Existing World-Class U.S. Government AI Institutions and Programs that are Key to the Trump Administration’s AI Agenda
- Recommendation 1: Support the NIST AI Safety Institute as a key pillar of American AI excellence.
- Recommendation 2: Expand the National Artificial Intelligence Research Resource from pilot to full program.
- Recommendation 3: Enhance transparency, accountability, and industry engagement by preserving the AI use case inventory.
- Recommendation 4: Propel U.S. Scientific and Security AI Leadership by Supporting AI and Computing at DOE.
Policy Proposals to Unleash AI Innovation
As artificial intelligence continues transforming industries and reshaping global competition, the United States must take bold, coordinated action to maintain its technological leadership. A multi-agency approach could include launching a National Initiative for AI Explainability, accelerating materials science discovery through AI-powered autonomous laboratories, creating AI-ready datasets for the life sciences, establishing a NIST Foundation to enhance public-private collaboration in AI research, and creating a National Security AI Entrepreneur Visa to attract and retain top global talent. Together, these initiatives would strengthen America’s AI ecosystem by addressing critical challenges in transparency, scientific research, standards development, and talent acquisition—while ensuring the U.S. remains at the forefront of responsible AI innovation.
Recommendation 1. Promote Innovation in Trustworthy AI through a Public-Private National Initiative for AI Explainability
Understanding the inner workings of AI systems is critical not only for reliability and risk mitigation in high-stakes areas such as defense, healthcare, and finance, but also for bolstering American technological leadership and maximizing government accountability and efficiency. However, despite promising progress in fields such as “mechanistic interpretability”, the study of explainability in AI systems is still nascent. A lack of explainability risks undermining trust and inhibiting AI adoption, particularly in safety-critical sectors.
To address the challenge of understanding and improving AI systems, we propose the launch of a Public-Private National Initiative for AI Explainability. Following in the footsteps of government-coordinated research projects like the Human Genome Project, this initiative would unite researchers, industry leaders, standards bodies, and government agencies to map the inner workings of advanced AI systems in a public-private partnership.
Federal precedent for such work already exists: DARPA’s 2017-2021 Explainable AI (XAI) program sought to create machine learning systems capable of explaining their decisions in a way humans could understand. While the program advanced techniques for explainable models and human-friendly translations of complex AI reasoning, the rapid development and scaling of AI technologies in the past five years demand a renewed, more ambitious effort.
The objectives of the initiative would include:
- Creating Open-Access Resources: Developing AI models, datasets, and tools accessible to researchers and practitioners, allowing a larger number of actors to contribute to progress.
- Developing Standardized Metrics and Benchmarks: Establishing clear standards to evaluate the explainability of AI systems in different circumstances, ensuring consistency and reliability across applications.
- Defining Common Tasks: Establishing standardized metrics and open datasets to create “common tasks” in explainability—well-defined challenges that drive innovation and encourage widespread progress as the broader ecosystem competes to improve performance.
- Investigating User-Centric Explanation Needs: Conducting research to identify which types of AI explanations are most effective and meaningful, and which provide appropriate degrees of control, to users across various contexts and applications.
- Developing a Repository of Explainability Techniques: Researching and disseminating explainability methods applicable across various AI domains, including an analysis of the strengths and weaknesses of different approaches and where they can be properly applied.
Implementation Strategy:
To launch this effort, the President should issue an executive order to signal national commitment and assign leadership to key federal agencies, including:
- Office of Science and Technology Policy: Playing a coordinating role across the government.
- Defense Advanced Research Projects Agency (DARPA): Building upon its prior experience with the XAI program to spearhead research efforts.
- National Institute of Standards and Technology (NIST): Developing standards and benchmarks for AI explainability, building on previous work in this area.
- National Science Foundation (NSF): Funding academic research through its National AI Research Institutes.
- Department of Energy (DOE): Leveraging its computational resources and expertise in large-scale research projects.
- Other government agencies with relevant expertise: For example, the National Institutes of Health (NIH) could focus on explainability in AI applications within the healthcare sector.
The White House should leverage its convening power to unite leading AI companies, top academic institutions, and government agencies in formal collaborations. These partnerships could encompass co-funded research, shared datasets and computing resources, collaborative access to advanced AI models, and joint development of open-source tools. Establishing a structured public-private partnership will facilitate coordinated funding, align strategic priorities, and streamline resource sharing, ensuring that advancements in AI explainability directly support both national interests and economic competitiveness. To sustain this initiative, the administration should also secure consistent, multi-year federal funding through appropriations requests to Congress.
DARPA’s XAI program showed that AI explainability requires interdisciplinary collaboration to align technical development with human understanding. Building on these insights, this initiative should include experts from computer science, cognitive science, ethics, law, and domain-specific fields to ensure explanations are clear, useful, and actionable for decision-makers across critical sectors.
By implementing this National Initiative for AI Explainability, the Trump administration can significantly enhance public confidence in AI technologies, accelerate responsible adoption by both the public and private sectors, and solidify America’s global leadership in AI innovation. Critically, a modest investment of government resources in this initiative could unlock substantial private-sector investment, spurring innovation and driving economic growth. This strategic approach will also enhance government accountability, optimize the responsible use of taxpayer resources, and ensure that American industry continues to lead in AI development and deployment.
Recommendation 2. Direct the Department of Energy (DOE) to use AI to Accelerate the Discovery of New Materials (link to full memo >>>)
Innovations in AI and robotics could revolutionize materials science by automating experimental processes and dramatically accelerating the discovery of new materials. Currently, materials science research involves manually testing different combinations of elements to identify promising materials, which limits the pace of discovery. Using AI foundation models for physics and chemistry, scientists could simulate new materials, while robotic “self-driving labs” could run 24/7 to synthesize and evaluate them autonomously. This approach would enable continuous data generation, refining AI models in a feedback loop that speeds up research and lowers costs. Given its expertise in supercomputing, AI, and a vast network of national labs, the Department of Energy (DOE) could lead this transformative initiative, potentially unlocking advancements in critical materials, such as improved battery components, that could have immense economic and technological impacts.
Recommendation 3. Create AI-ready Collaborative Datasets to Accelerate Progress in the Life Sciences (link to full memo >>>)
Large, high-quality datasets could revolutionize life science research by powering AI models that unlock new discoveries in areas like drug development and diagnostics. Currently, researchers often work in silos with limited incentives to collaborate and share meticulously curated data, slowing progress. By launching a government-funded, end-to-end initiative—from identifying critical dataset needs to certifying automated collection methods and hosting robust open repositories—scientists could continuously generate and refine data, fueling AI models in a feedback loop that boosts accuracy and lowers costs. Even a relatively modest government investment could produce vital resources for researchers and startups to spark new industries. This model could also be extended to a range of other scientific fields to accelerate U.S.science and innovation.
Recommendation 4. Create a NIST Foundation to Support the Agency’s AI Mandate (link to full memo >>>)
To maintain America’s competitive edge in AI, NIST needs greater funding, specialized talent, and the flexibility to work effectively with private-sector partners. One solution is creating a “NIST Foundation,” modeled on the DOE’s Foundation for Energy Security and Innovation (FESI), which combines federal and private resources to expand capacity, streamline operations, and spur innovation. Legislation enabling such a foundation was introduced with bipartisan support in the 118th Congress, signaling broad consensus on its value. The Trump administration can direct NIST to study how a nonprofit foundation might boost its AI initiatives and broader mission—just as a similar report helped pave the way for FESI—giving Congress the evidence it needs to formally authorize a NIST Foundation. The administration can also support passage of authorizing legislation through Congress.
Recommendation 5. Attract Top Global Talent by Creating a National Security AI Entrepreneur Visa for Elite Dual-use Technology Founders (link to full memo >>>)
America’s leadership in AI has been driven by the contributions of immigrant entrepreneurs, with companies like NVIDIA, Anthropic, OpenAI, X, and HuggingFace—all of which have at least one immigrant co-founder—leading the charge. To maintain this competitive edge as global competition intensifies, the administration should champion a National Security Startup Visa specifically targeted at high-skilled founders of AI firms. These entrepreneurs are at the forefront of developing dual-use technologies critical for both America’s economic leadership and national security. Although the linked proposal above is targeted at legislative action, the administration can take immediate steps to advance this priority by publicly supporting legislation to establish such a visa, engaging with Congressional allies to underscore its strategic importance, and directing agencies like the Department of Homeland Security and the Department of Commerce to explore ways to streamline pathways for these innovators. This decisive action would send a clear signal that America remains the destination of choice for world-class talent, ensuring that the nation stays ahead in the race for AI dominance.
Policy Proposals to Accelerate AI Adoption Across the Economy
AI has transformative potential to boost economic growth and unlock new levels of prosperity for all. The Trump administration should take bold action to encourage greater adoption of AI technologies and AI expertise by leveraging government procurement, hiring, and standards-setting processes, alongside coordinated support for America’s teachers to prepare students to join the future AI workforce. In government, a coordinated set of federal initiatives is needed to modernize and streamline effective AI adoption in the public sector. These proposals include developing a national digital platform through GSA to streamline AI procurement processes, establishing a federal center of excellence to support state and local governments in AI implementation, and pursuing innovative hiring models to expand AI expertise at HHS. Additionally, NIST should develop voluntary standards for measuring AI energy and resource usage to inform infrastructure planning efforts. Finally, the President should announce a national teacher talent surge and set AI as a competitive priority in American education.
Recommendation 1. Streamline Procurement Processes for Government Use of AI (link to full memo >>>)
The federal government has a critical role in establishing standards for AI systems to enhance public services while ensuring they are implemented ethically and transparently. To streamline this effort and support federal agencies, the administration should direct the General Services Administration (GSA) to create a user-friendly, digital platform for AI procurement. This platform would simplify the acquisition process by providing agencies with clear, up-to-date guidelines, resources, and best practices, all tailored to align with existing procurement frameworks. The platform would empower agencies to make informed decisions that prioritize safety, fairness, and effective use of AI technologies, while demonstrating the administration’s commitment to modernizing government operations and ensuring America leads the way in adopting cutting-edge AI solutions.
Recommendation 2. Establish a Federal Center of Excellence to Expand State and Local Government Capacity for AI Procurement and Use (link to full memo >>>)
State and local governments often face challenges in effectively leveraging AI to enhance their efficiency and service capabilities. To support responsible AI adoption at the state, local, tribal, and territorial (SLTT) levels, the administration should establish a federal AI Center of Excellence. This center would provide hands-on guidance from experts in government, academia, and civil society, helping SLTT agencies navigate complex challenges such as limited technical expertise, budget constraints, privacy concerns, and evolving regulations. It would also translate existing federal AI standards—including Executive Order 13960 and the NIST Risk Management Framework—into practical, actionable advice. By developing in-house procurement and deployment expertise, SLTT governments could independently and confidently implement AI solutions, promoting innovation while ensuring responsible, effective, and efficient use of taxpayer resources.
Recommendation 3. Pilot an AI Corps at HHS to Drive Government-Wide AI Adoption (link to full memo >>>)
Federal agencies often struggle to leverage AI effectively, due to limited technical expertise and complex oversight requirements. Modeled after the Department of Homeland Security’s successful AI Corps, which has improved disaster response and cybersecurity, this pilot would embed AI and machine learning experts within the Department of Health and Human Services’s (HHS) 10 agencies, accelerating responsible AI implementation in healthcare, driving greater efficiency, and demonstrating a scalable model that could be replicated across other federal departments. HHS is uniquely suited for piloting an AI Corps because it oversees critical health infrastructure and massive, sensitive datasets—presenting significant opportunities for AI-driven improvements but also requiring careful management. If successful, this pilot could serve as a strategic blueprint to enhance AI adoption, improve government performance, and maximize the responsible use of taxpayer resources across the federal government.
Recommendation 4. Make America’s Teacher Workforce Competitive for the AI Era (link to full memo >>>)
With America facing a significant shortage of teachers and a growing need for AI and digital skills in the workforce, the Trump administration can rebuild America’s teaching profession by launching a coordinated strategy led by the Office of Science and Technology Policy (OSTP). This initiative should begin with a national teacher talent surge to expand annual teacher graduates by 100,000, addressing both the urgent workforce gap and the imperative to equip students for an AI-driven future. The plan includes a Challenge.gov competition to attract innovative recruitment and retention models, updating Department of Education scholarship programs (like the Graduate Assistance in Areas of National Need) to include AI, data science, and machine learning, convening colleges of education to modernize training, and directing agencies to prioritize AI-focused teacher development. By leveraging existing grants (e.g., Teacher Quality Partnerships, SEED, the STEM Corps, and Robert Noyce Scholarships), the administration can ensure a robust pipeline of educators ready to guide the next generation.
Recommendation 5. Prepare U.S. Energy Infrastructure for AI Growth Through Standardized Measurement and Forecasting
As AI adoption accelerates, America’s energy infrastructure faces a critical challenge: next-generation AI systems could place unprecedented demands on the power grid, yet the lack of standardized measurements, and wide variations in forecasted demand, leaves utilities and policymakers unprepared. Without proactive planning, energy constraints could slow AI innovation and undermine U.S. competitiveness.
To address this, the Administration should direct the National Institute of Standards and Technology (NIST) and the Department of Energy (DOE) to develop a standardized framework for measuring and forecasting AI’s energy and resource demands. This framework should be paired with a voluntary reporting program for AI developers—potentially collected by the Energy Information Administration (EIA)—to provide a clearer picture of AI’s impact on energy consumption. The EIA should also be tasked with forecasting AI-driven energy demand, ensuring that utilities, public utility commissions, and state energy planners have the data needed to modernize the grid efficiently.
Greater transparency will enable both government and industry to anticipate energy needs, drive investment in grid modernization, and prevent AI-related power shortages that could hinder economic growth. The proactive integration of AI and energy planning will strengthen America’s leadership in AI innovation while safeguarding the reliability of its infrastructure. FAS is actively developing policy proposals with the science and technology community at the intersection of AI and energy. We plan to share additional recommendations on this topic in the coming months.
Policy Proposals to Ensure Secure and Trustworthy AI
Privacy
Protecting Americans’ privacy while harnessing the potential of AI requires decisive federal action that prioritizes both individual rights and technological advancement. Strengthening privacy protections while enabling responsible data sharing is crucial for ensuring that AI-driven innovations improve public services without compromising sensitive information. Key initiatives include establishing NIST-led guidelines for secure data sharing and maintaining data integrity, implementing a FedRAMP authorization framework for third-party data sources used by government agencies, and promoting the use of Privacy Enhancing Technologies (PETs). Additionally, the administration should create a “Responsible Data Sharing Corps” to provide agencies with expert guidance and build capacity in responsible data practices.
Recommendation 1. Secure Third Party Commercial Data for AI through FedRAMP Authorization (link to full memo >>>)
The U.S. government is a major customer of commercial data brokers and should require a pre-evaluation process before agencies acquire large datasets, ensuring privacy and security from the outset. Thoroughly vetting data brokers and verifying compliance standards can help avert national security risks posed by compromised or unregulated third-party vendors. To formalize these safeguards, OMB and FedRAMP should create an authorization framework for data brokers that provide commercially available information, especially with personally identifiable information. Building on its established role in securing cloud providers FedRAMP is well positioned to guide these protocols, ensuring agencies work only with trusted vendors and strengthening overall data protection.
Recommendation 2. Catalyze Federal Data Sharing through Privacy Enhancing Technologies (link to full memo >>>)
To maintain America’s leadership in AI and digital innovation, the administration must ensure that government agencies can securely leverage data while protecting privacy and maintaining public trust. The federal government can lead by example through the adoption of Privacy Enhancing Technologies (PETs)—tools that enable data analysis while minimizing exposure of sensitive information. Agencies should be encouraged to adopt PETs with support from a Responsible Data Sharing Corps, while NIST develops a decision-making framework to guide their use. OMB should require agencies to apply this framework in data-sharing initiatives and report on PET adoption, with a PET Use Case Inventory and annual reports enhancing transparency. A federal fellowship program could also bring in experts from academia and industry to drive PET innovation. These measures would strengthen privacy, security, and public trust while positioning the U.S. as a global leader in responsible data use.
Recommendation 3. Establish Data-Sharing Standards to Support AI Development in Healthcare (link to full memo >>>)
The U.S. healthcare system generates vast amounts of data daily, yet fragmentation, privacy concerns, and lack of interoperability severely limit its use in AI development, hindering medical innovation. To address this, the AI Action Plan should direct NIST to lead an interagency coalition in developing standardized protocols for health data anonymization, secure sharing, and third-party access. By establishing clear technical and governance standards—similar to NIST’s Cryptographic and Biometric Standards Programs—this initiative would enable responsible research while ensuring compliance with privacy and security requirements. These standards would unlock AI-driven advancements in diagnostics, treatment planning, and health system efficiency. Other nations, including the U.K., Australia, and Finland, are already implementing centralized data-sharing frameworks; without federal leadership, the U.S. risks falling behind. By taking decisive action, the administration can position the U.S. as a global leader in medical AI, accelerating innovation while maintaining strong privacy protections.
Security, Safety, and Trustworthiness
AI holds immense promise for job growth, national security, and innovation, but accidents or misuse risk undermining public trust and slowing adoption—threatening the U.S.’s leadership in this critical field. The following proposals use limited, targeted government action alongside private-sector collaboration to strengthen America’s AI capabilities while upholding public confidence and protecting our national interests.
Recommendation 1. Establish an Early Warning System for AI-Powered Threats to National Security and Public Safety (link to full memo >>>)
Emerging AI capabilities could also pose severe threats to public safety and national security. AI companies are already evaluating their most advanced models to identify dual-use capabilities, such as the capacity to conduct offensive cyber operations, enable the development of biological or chemical weapons, and autonomously replicate and spread. These capabilities can arise unpredictably and undetected during development and after deployment. To prepare for these emerging risks, the federal government should establish a coordinated “early-warning system” for novel dual-use AI capabilities to gain awareness of emerging risks before models are deployed. A government agency could serve as a central information clearinghouse—an approach adapted from the original congressional proposal linked above. Advanced AI model developers could confidentially report newly discovered or assessed dual-use capabilities, and the White House could direct relevant government agencies to form specialized working groups that engage with private sector and other non-governmental partners to rapidly mitigate risks and leverage defensive applications. This initiative would ensure that the federal government and its stakeholders have maximum lead time to prepare for emerging AI-powered threats, positioning the U.S. as a leader in safe and responsible AI innovation.
Recommendation 2. Create a Voluntary AI Incident Reporting Hub to Monitor Security Incidents from AI (link to full memo >>>)
The federal government should establish a voluntary national Artificial Intelligence Incident Reporting Hub to better track, analyze, and address incidents from increasingly complex and capable AI systems that are deployed in the real world. Such an initiative could be modeled after successful incident reporting and info-sharing systems operated by the National Cybersecurity FFRDC, the Federal Aviation Administration, and the Food and Drug Administration. By providing comprehensive yet confidential data collection under the umbrella of an agency (e.g. NIST) this initiative would bolster public trust, facilitate the sharing of critical risk information, and enable prompt government action on emerging threats, from cybersecurity vulnerabilities to potential misuse of AI in sensitive areas like chemical, biological, radiological, or nuclear contexts. This proposal builds on bipartisan legislation introduced in the last Congress, as well as the memo linked above, which was originally targeted at Congressional action.
Recommendation 3. Promote AI Trustworthiness by Providing a Safe Harbor for AI Researchers (link to full memo >>>)
Independent AI research plays a key role in ensuring safe and reliable AI systems. In 2024, over 350 researchers signed an open letter calling for “a safe harbor for independent AI evaluation”, noting that generative AI companies offer no legal protections for independent safety researchers. This situation is unlike established voluntary protections from companies for traditional software, and Department of Justice (DOJ) guidance not to prosecute good faith security research. The proposal linked above was targeted at Congressional action, however the executive branch could adapt these ideas in several ways, by, for example: 1) instructing the Office of Management and Budget (OMB) to issue guidance to all federal agencies requiring that contracting documents for generative AI systems include safe-harbor provisions for good-faith external research, consistent with longstanding federal policies that promote responsible vulnerability disclosure. 2) Coordinating with DOJ and relevant agencies to clarify that good-faith AI security and safety testing—such as red-teaming and adversarial evaluation—does not violate the Computer Fraud and Abuse Act (CFAA) or other laws when conducted according to established guidelines.
Recommendation 4. Build a National Digital Content Authentication Technologies Research Ecosystem (link to full memo >>>)
AI generated synthetic content (such as fake videos, images, and audio) is increasingly used by malicious actors to defraud elderly Americans, spread child sexual abuse material, and impersonate political figures. To counter these threats, the United States must invest in developing technical solutions for reliable synthetic content detection. Through the National Institute of Standards and Technology (NIST), the Trump Administration can: 1) establish dedicated university-led national research centers, 2) develop a national synthetic content database, and 3) run and coordinate prize competitions to strengthen technical countermeasures.These initiatives will help build a robust research ecosystem to keep pace with the rapidly evolving synthetic content threat landscape, maintaining America’s role as a global leader in responsible and secure AI.
Recommendation 5. Strengthen National Security by Evaluating AI-Driven Biological Threats (link to full memo >>>)
Over the past two years, the rapid advance of AI in biology and large language models has highlighted an urgent need for a targeted U.S. Government program to assess and mitigate biosecurity risks. While AI-enabled tools hold immense promise for drug discovery, vaccine research, and other beneficial applications, their dual-use potential (e.g., identifying viral mutations that enhance vaccine evasion) makes them a national security priority. Building on the Department of Homeland Security’s (DHS) previous work on AI and CBRN threats, the Department of Energy (DOE), DHS, and other relevant agencies, should now jointly launch a “Bio Capability Evaluations” program, backed by sustained funding, to develop specialized benchmarks and standards for evaluating dangerous biological capabilities in AI-based research tools. By forming public-private partnerships, creating a DOE “sandbox” for ongoing testing, and integrating results into intelligence assessments, such a program would enable more nuanced, evidence-based regulations and help the United States stay ahead of potential adversaries seeking to exploit AI’s biological capabilities.
Policy Proposals to Strengthen Existing World-Class U.S. Government AI Institutions and Programs that are Key to the Trump Administration’s AI Agenda
A robust institutional framework is essential for ensuring that the government fulfills its role in AI research, industry coordination, and ecosystem development. The previous Trump administration laid the groundwork for American AI leadership, and the institutions established since then can be leveraged to further assert U.S. dominance in this critical technological space.
Recommendation 1. Support the NIST AI Safety Institute as a Key Pillar of American AI Excellence
The NIST AI Safety Institute (AISI) has assembled a world-leading team to ensure that the U.S. leads in safe, reliable, and trustworthy AI development. As AI integrates into critical sectors like national security, healthcare, and finance, strong safety standards are essential. AISI develops rigorous benchmarks, tests model security, and collaborates with industry to set standards, mitigating risks from unreliable AI. Strengthening AISI protects U.S. consumers, businesses, and national security while boosting global trust in the U.S. AI ecosystem—enhancing international adoption of American AI models. AISI has broad support, with bipartisan legislation to codify the AISI advanced in Congress and backing from organizations across industry and academia. The AI Action Plan should prioritize AISI as a pillar of AI policy.
Recommendation 2. Expand the National Artificial Intelligence Research Resource from Pilot to Full Program
For decades, academic researchers have driven AI breakthroughs, laying the foundation for the technologies that now shape global competition. However, as AI development becomes increasingly concentrated within large technology companies, the U.S. risks losing the ecosystem that made these advances possible. The National AI Research Resource (NAIRR) Pilot is a critical initiative to keep American AI innovation competitive and accessible. By providing researchers and educators across the country access to cutting-edge AI tools, datasets, and computing power, NAIRR ensures that innovation is not confined to a handful of dominant firms but widely distributed. To keep America at the forefront of AI, the Trump Administration should expand NAIRR into a full-fledged program. Allowing the program to lapse would erode America’s leadership in AI research, forcing top talent to seek resources elsewhere. To secure its future, the White House should support bipartisan legislation to fully authorize NAIRR and include it in the President’s Budget Request, ensuring sustained investment in this vital initiative.
Recommendation 3. Enhance Transparency, Accountability, and Industry Engagement by Preserving the AI Use Case Inventory (link to letter of support >>>)
The AI Use Case Inventory, established under President Trump’s Executive Order 13960 and later codified in section 7225 of the FY23 National Defense Authorization Act, plays a crucial role in fostering public trust and innovation in government AI use. Recent OMB guidance (M-24-10) has expanded its scope, refining AI classifications and standardizing AI definitions. The inventory enhances public trust and accountability by ensuring transparency in AI deployments, tracks AI successes and risks to improve government services, and supports AI vendors by providing visibility into public-sector AI needs, thereby driving industry innovation. As the federal government considers revisions to M-24-10 and its plan for AI adoption within federal agencies, OMB should uphold the 2024 guidance on federal agency AI Use Case Inventories and ensure agencies have the necessary resources to complete it effectively.
Recommendation 4. Propel U.S. Scientific and Security AI Leadership by Supporting AI and Computing at DOE
The Department of Energy (DOE) hosts leading research and innovation centers, particularly under the Undersecretary for Science and Innovation. The Office of Critical and Emerging Technologies (CET), for example, plays a key role in coordinating AI initiatives, including the proposed Frontiers in Artificial Intelligence for Science, Security, and Technology (FASST) program. To fully harness AI’s potential, DOE should establish a dedicated AI and Computing Laboratory under the Undersecretary, ensuring a strategic, mission-driven approach to AI development. This initiative would accelerate scientific discovery, strengthen national security, and tackle energy challenges by leveraging DOE’s advanced computational infrastructure and expertise. To ensure success, it should be supported by a multi-year funding commitment and flexible operational authorities, modeled after ARPA-E, to streamline hiring, procurement, and industry-academic partnerships.
Conclusion
These recommendations offer a roadmap for securing America’s leadership in artificial intelligence while upholding the fundamental values of innovation, competitiveness, and trustworthiness. By investing in cutting-edge research, equipping government and educators with the tools to navigate the AI era, and ensuring safety, the new administration can position America as a global standard-bearer for trustworthy and effective AI development.
Public Comment on Executive Branch Agency Handling of CAI containing PII
Public comments serve the executive branch by informing more effective, efficient program design and regulation. As part of our commitment to evidence-based, science-backed policy, FAS staff leverage public comment opportunities to embed science, technology, and innovation into policy decision-making.
The Federation of American Scientists (FAS) is a non-partisan, nonprofit organization committed to using science and technology to benefit humanity by delivering on the promise of equitable and impactful policy. FAS believes that society benefits from a federal government that harnesses science, technology, and innovation to meet ambitious policy goals and deliver impactful results to the public.
We are writing in response to your Request for Information on the Executive Branch Agency Handling of Commercially Available Information (CAI) Containing Personally Identifiable Information (PII). Specifically, we will be answering questions 2 and 5 in your request for information:
2. What frameworks, models, or best practices should [the White House Office of Management and Budget] consider as it evaluates agency standards and procedures associated with the handling of CAI containing PII and considers potential guidance to agencies on ways to mitigate privacy risks from agencies’ handling of CAI containing PII?
5. Agencies provide transparency into the handling of PII through various means (e.g., policies and directives, Privacy Act statements and other privacy notices at the point of collection, Privacy Act system of records notices, and privacy impact assessments). What, if any, improvements would enhance the public’s understanding of how agencies handle CAI containing PII?
Background
In the digital landscape, commercially available information (CAI) represents a vast ecosystem of personal data that can be easily obtained, sold, or licensed to various entities. The Executive Order on Safe, Secure, and Trustworthy Development and Use of Artificial Intelligence (EO 14110) defines CAI comprehensively as information about individuals or groups that is publicly accessible, encompassing details like device information and location data.
A 2017 report by the Georgetown Law Review found that 63% of Americans can be uniquely identified using just three basic attributes—gender, birth date, and ZIP code—with an astonishing 99.98% of individuals potentially re-identifiable from a dataset containing only 15 fundamental characteristics. This vulnerability underscores the critical challenges of data privacy in an increasingly interconnected world.
CAI takes on heightened significance in the context of artificial intelligence (AI) deployment, as these systems enable both data collection and the use of advanced inference models to analyze datasets and produce predictions, insights, and assumptions that reveal patterns or relationships not directly evident in the data. Some AI systems can allow the intentional or unintentional reidentification of supposedly anonymized private data. These capabilities raise questions about privacy, consent, and the potential for unprecedented levels of personal information aggregation and analysis, challenging existing data protection frameworks and individual rights.
The United States federal government is one of the largest customers of commercial data brokers. Government entities increasingly use CAI to empower public programs, enabling federal agencies to augment decision-making, policy development, and resource allocation and enrich research and innovation goals with large yet granular datasets. For example, the National Institutes of Health have discussed within their data strategies how to incorporate commercially available data into research projects. The use of commercially available electronic health records is essential for understanding social inequalities within the healthcare system but includes sensitive personal data that must be protected.
However, government agencies face significant public scrutiny over their use of CAI in areas including law enforcement, homeland security, immigration, and tax administration. This scrutiny stems from concerns about privacy violations, algorithmic bias, and the risks of invasive surveillance, profiling, and discriminatory enforcement practices that could disproportionately harm vulnerable populations. For example, federal agencies like Immigration and Customs Enforcement (ICE) and Customs and Border Protection (CBP) have used broker-purchased location data to track individuals without warrants, raising constitutional concerns.
In 2020, the American Civil Liberties Union filed a Freedom of Information Act lawsuit against several Department of Homeland Security (DHS) agencies, arguing that the DHS’s use of cellphone data and data from smartphone apps constitutes unreasonable searches without a warrant and violates the Fourth Amendment. A report by the Electronic Frontier Foundation found that CAI was used for mass surveillance practices, including geofence warrants that query all phones in specific locations, further challenging constitutional protections.
While the Privacy Act of 1974 covers the use of federally collected personal information by agencies, there is no explicit guidance governing federal use of third-party data. The bipartisan Fourth Amendment is Not for Sale Act (H.R.4639) would bar certain technology providers—such as remote computing service and electronic communication service providers—from sharing the contents of stored electronic communications with anyone (including government actors) and from sharing customer records with government agencies. The bill has passed the House of Representatives in the 118th Congress but has yet to pass the Senate as of December 2024. Without protections in statute, it is imperative that the federal government crafts clear guidance on the use of CAI containing PII in AI systems. In this response to the Office of Management and Budget’s (OMB) request for information, FAS will outline three policy ideas that can improve how federal agencies navigate the use of CAI containing PII, including in AI use.
Summary of Recommendations
The federal government is responsible for ensuring the safety and privacy of the processing of personally identifiable information within commercially available information used for the development and deployment of artificial intelligence systems. For this RFI, FAS brings three proposals to increase government capacity in ensuring transparency and risk mitigation in how CAI containing PII is used, including in agency use of AI:
- Enable FedRAMP to Create an Authorization System for Third-Party Data Sources: An authorization framework for CAI containing PII would ensure a standardized approach for data collection, management, and contracting, mitigating risks, and ensuring ethical data use.
- Expand Existing Privacy Impact Assessments (PIA) to Incorporate Additional Requirements and Periodic Evaluations: Regular public reports on CAI sources and usage will enable stakeholders to monitor federal data practices effectively.
- Build Government Capacity for the Use of Privacy Enhancing Technologies to Bolster Anonymization Techniques by harnessing existing resources such as the United States Digital Service (USDS).
Recommendation 1. Enable FedRAMP to Create an Authorization System for Third-Party Data Sources
Government agencies utilizing CAI should implement a pre-evaluation process before acquiring large datasets to ensure privacy and security. OMB, along with other agencies that are a part of the governing board of the Federal Risk and Authorization Management Program (FedRAMP), should direct FedRAMP to create an authorization framework for third-party data sources that contract with government agencies, especially data brokers that provide CAI with PII, to ensure that these vendors comply with privacy and security requirements. FedRAMP is uniquely positioned for this task because of its previous mandate to ensure the safety of cloud service providers used by the federal government and its recent expansion of this mandate to standardize AI technologies. The program could additionally harmonize its new CAI requirements with its forthcoming AI authorization framework.
When designing the content of the CAI authorization, a useful benchmark in terms of evaluation criteria is the Ag Data Transparent (ADT) certification process. Companies applying for this certification must submit contracts and respond to 11 data collection, usage, and sharing questions. Like the FedRAMP authorization process, a third-party administrator reviews these materials for consistency, granting the ADT seal only if the company’s practices align with its contracts. Any discrepancies must be corrected, promoting transparency and protecting farmers’ data rights. The ADT is a voluntary certification, and therefore does not provide a good model for enforcement. However, it does provide a framework for the kind of documentation that should be required. The CAI authorization should thus include the following information required by the ADT certification process:
- Data source: The origin or provider of the data, such as a specific individual, organization, database, device, or system, that supplies information for analysis or processing, as well as the technologies, platforms, or applications used to collect data. For example, the authorization framework should identify if an AI system collected, compiled, or aggregated a CAI dataset.
- Data categories: The classification of data based on its format or nature, such as structured (e.g., spreadsheets), unstructured (e.g., text or images), personal (e.g., names, Social Security numbers), or non-personal (e.g., aggregated statistics).
- Data ownership: A description of any agreements in place that define which individual or organization owns the data and what happens when that ownership is transferred.
- Third-party data collection contractors: An explanation of whether or not partners or contractors associated with the vendor have to follow the company’s data governance standards.
- Consent and authorization to sell to third-party contractors: A description of whether or not there is an explicit agreement between data subjects (e.g., an individual using an application) that their data can be collected and sold to the government or another entity for different purposes, such as use to train or deploy an AI system. In addition, a description of the consent that has been obtained for that use.
- Opt out and deletion: Whether or not the data can be deleted at the request of a data subject, or if the data subject opt out of certain data use. A description of the existing mechanisms where individuals can decline or withdraw consent for their data to be collected, processed, or used, ensuring they retain control over their personal information.
- Security safeguards and breach notifications: The measures and protocols implemented to protect data from unauthorized access, breaches, and misuse. These include encryption, access controls, secure storage, vulnerability testing, and compliance with industry security standards.
Unlike the ADT, a FedRAMP authorization process can be strictly enforced. FedRAMP is mandatory for all cloud service providers working with the executive branch and follows a detailed authorization process with evaluations and third-party auditors. It would be valuable to bring that assessment rigor to federal agency use of CAI, and would help provide clarity to commercial vendors.
The authorization framework should also document the following specific protocols for the use of CAI within AI systems:
- Provide a detailed explanation of which datasets were aggregated and the efforts to minimize data. According to a report by the Information Systems Audit and Control Association (ISACA), singular data points, when combined, can compromise anonymity, especially when placed through an AI system with inference capabilities.
- Type of de-identification or anonymization technique used. Providing this information helps agencies assess whether additional measures are necessary, particularly when using AI systems capable of recognizing patterns that could re-identify individuals.
By setting these standards, this authorization could help agencies understand privacy risks and ensure the reliability of CAI data vendors before deploying purchased datasets within AI systems or other information systems, therefore setting them up to create appropriate mitigation strategies.
By encouraging data brokers to follow best practices, this recommendation would allow agencies to focus on authorized datasets that meet privacy and security standards. Public availability of this information could drive market-wide improvements in data governance and elevate trust in responsible data usage. This approach would support ethical data governance in AI projects and create a more transparent, publicly accountable framework for CAI use in government.
Recommendation 2. Expand Privacy Impact Assessments (PIA) to Incorporate Additional Requirements and Periodic Evaluations
Public transparency regarding the origins and details of government-acquired CAI containing PII is critical, especially given the largely unregulated nature of the data broker industry at the federal level. Privacy Impact Assessments (PIAs) are mandated under Section 208 of the 2002 E-Government Act and OMB Memo M-03-22, and can serve as a vital policy tool for ensuring such transparency. Agencies must complete PIAs at the outset of any new electronic information collection process that includes “information in identifiable form for ten or more persons.” Under direction from Executive Order 14110 on Safe, Secure, and Trustworthy Development and Use of Artificial Intelligence, OMB issued a request for information in April 2024 to explore updating PIA guidance for AI-era privacy concerns, although new guidance has not yet been issued.
To ensure that PIAs can effectively provide transparency into government practices on CAI that contains PII, we recommend that OMB provide updated guidance requiring agencies to regularly review and update their PIAs at least every three years, and also require agencies to report more comprehensive information in PIAs. We provide more details on these recommendations below.
First, OMB should guide agencies to periodically update their PIAs to ensure evolutions in agency data practices are publicly captured, which is increasingly important as data-driven AI systems are adopted by government actors and create novel privacy concerns. Under OMB Memo M-03-22, agencies must initiate or update PIAs when new privacy risks or factors emerge that affect the collection and handling of PII, including when agencies incorporate PII obtained from commercial or public sources into existing information systems. However, a public comment submitted by the Electronic Privacy Information Center (EPIC) pointed out that many agencies fail to publish and update required PIAs in a timely manner, indicating that a stricter schedule is needed to maintain accountability for PIA reporting requirements. As data privacy risks evolve through the advancement of AI systems, increased cybersecurity risks, and new legislation, it is essential that a minimum standard schedule for updating PIAs is created to ensure agencies provide the public with an up-to-date understanding of the potential risks resulting from using CAI that includes PII. For example, the European Union’s General Data Protection Regulation (Art. 35) requires PIAs to be reconducted every three years.
Second, agency PIAs should report more detailed information on the CAI’s source, vendor information, contract agreements, and licensing arrangements. A frequent critique of existing PIAs is that they contain too little information to inform the public of relevant privacy harms. Such a lack of transparency risks damaging public trust in government. One model for expanded reporting frameworks for CAI containing PII is the May 2024 Policy Framework for CAI, established for the Intelligence Community (IC) by the Office of the Director of National Intelligence (ODNI). This framework requires the IC to document and report “the source of the Sensitive CAI and from whom the Sensitive CAI was accessed or collected” and “any licensing agreements and/or contract restrictions applicable to the Sensitive CAI”. OMB should incorporate these reporting practices into agency PIA requirements and explicitly require agencies to identify the CAI data vendor in order to provide insight into the source and quality of purchased data.
Many of these elements are also present in Recommendation 1, for a new FedRAMP authorization framework. However, that recommendation does not include existing agency projects using CAI or agencies that could contract CAI datasets outside of the FedRAMP authorization. Including this information within the PIA framework also allows for an iterative understanding of privacy risks throughout the lifecycle of a project using CAI.
By obligating agencies to provide more frequent PIA updates and include additional details on the source, vendor, contract and licensing arrangements for CAI containing PII, the public gains valuable insight into how government agencies acquire, use, and manage sensitive data. These updates to PIAs would allow civil society groups, journalists, and other external stakeholders to track government data management practices over time during this critical juncture where federal uptake of AI systems is rapidly increasing.
Recommendation 3. Build Government Capacity for the Use of Privacy Enhancing Technologies to Bolster Anonymization Techniques
Privacy Enhancing Technologies (PETs) are a diverse set of tools that can be used throughout the data lifecycle to ensure privacy by design. They can also be powerful tools in ensuring that PII within CAI) is adequately anonymized and secure. OMB should collect information on current agency PET usage, gather best practices, and identify deployment gaps. To address these gaps, OMB should collaborate with agencies like the USDS to establish capacity-building programs, leveraging initiatives like the proposed “Responsible Data Sharing Core” to provide expert consultations and enhance responsible data-sharing practices.
Meta’s Open Loop project identified eight types of PETs that are ripe to be deployed in AI systems, categorizing them into maturity levels, context of deployment, and limitations. One type of PET is differential privacy, a mathematical framework designed to protect individuals’ privacy in datasets by introducing controlled noise to the data. This ensures that the output of data analysis or AI models does not reveal whether a specific individual’s information is included in the dataset. The noise is calibrated to balance privacy with data utility, allowing meaningful insights to be derived without compromising personal information. Differential privacy is particularly useful in AI models that rely on large-scale data for training, as it prevents the inadvertent exposure of PII during the learning process. Within the federal government, the U.S. Census Bureau is using differential privacy to anonymize data while preserving its aggregate utility, ensuring compliance with privacy regulations and reducing re-identification within datasets.
Scaling the use of PETs in other agencies has been referenced in several U.S. government strategy documents, such as the National Strategy to Advance Privacy-Preserving Data Sharing and Analytics, which encourages federal agencies to adopt and invest in the development of PETs, and the Executive Order (EO) on the Safe, Secure, and Trustworthy Development and Use of Artificial Intelligence, which calls for federal agencies to identify where they could use PETs. As a continuation of this EO, the National Science Foundation and the Department of Energy established a Research Coordination Network on PETs that will “address the barriers to widespread adoption of PETs, including regulatory considerations.”
Although the ongoing research and development of PETS is vital to this growing field, there is an increasing need to ensure these technologies are implemented across the federal government. To kick this off, OMB should collect detailed information on how agencies currently use PETs, especially in projects that use CAI containing PII. This effort should include gathering best practices from agencies with successful PET implementations, such as the previous U.S. Census Bureau’s use of differential privacy. Additionally, OMB should identify gaps in PET deployment, assessing barriers such as technical capacity, funding, and awareness of relevant PETs. To address these gaps, OMB should collaborate with other federal agencies to design and implement capacity-building programs, equipping personnel with the knowledge and tools needed to integrate PETs effectively. For example, a forthcoming FAS’ Day One Project publication, “Increasing Responsible Data Sharing Capacity throughout Government,” seeks to harness existing government capabilities to build government capacity in deploying PETs. This proposal aims to enhance responsible data sharing in government by creating a capacity-building initiative called the “Responsible Data Sharing Core” (RDSC). Managed by the USDS, the RDSC would deploy fellows and industry experts to agencies to consult on data use and sharing decisions and offer consultations on which PETs are appropriate for different contexts.
Conclusion
The federal government’s increasing reliance on CAI containing PII presents significant privacy challenges. The current landscape of data procurement and AI deployment by agencies like ICE, CBP, and others raises critical concerns about potential Fourth Amendment violations, discriminatory profiling, and lack of transparency.
The ideas proposed in this memo—implementing FedRAMPamp authorization for data brokers, expanding privacy impact assessment requirements, and developing capacity-building programs for privacy-enhancing technologies—represent crucial first steps in addressing these systemic risks. As AI systems become increasingly integrated into government processes, maintaining a delicate balance between technological advancement and fundamental constitutional protections will be paramount to preserving individual privacy, promoting responsible adoption, and maintaining public trust.
We appreciate the opportunity to contribute to this Request for Information on Executive Branch Agency Handling of Commercially Available Information Containing Personally Identifiable Information. Please contact clangevin@fas.org if you have any questions or need additional information.
Public Comment on the U.S. Artificial Intelligence Safety Institute’s Draft Document: NIST AI 800-1, Managing Misuse Risk for Dual-Use Foundation Models
Public comments serve the executive branch by informing more effective, efficient program design and regulation. As part of our commitment to evidence-based, science-backed policy, FAS staff leverage public comment opportunities to embed science, technology, and innovation into policy decision-making.
The Federation of American Scientists (FAS) is a non-partisan organization dedicated to using science and technology to benefit humanity through equitable and impactful policy. With a strong track record in AI governance, FAS has actively contributed to the development of AI standards and frameworks, including providing feedback on NIST AI 600-1, the Generative AI Profile. Our work spans advocating for federal AI testbeds, recommending policy measures for frontier AI developers, and evaluating industry adoption of the NIST AI Risk Management Framework. We are members of the U.S. AI Safety Institute Research Consortium, and we responded to NIST’s request for information earlier this year concerning its responsibilities under sections 4.1, 4.5, and 11 of the AI Executive Order.
We commend NIST’s U.S. Artificial Intelligence Safety Institute for developing the draft guidance on “Managing Misuse Risk for Dual-Use Foundation Models.” This document represents a significant step toward establishing robust practices for mitigating catastrophic risks associated with advanced AI systems. The guidance’s emphasis on comprehensive risk assessment, transparent decision-making, and proactive safeguards aligns with FAS’s vision for responsible AI development.
In our response, we highlight several strengths of the guidance, including its focus on anticipatory risk assessment and the importance of clear documentation. We also identify areas for improvement, such as the need for harmonized language and more detailed guidance on model development safeguards. Our key suggestions include recommending a more holistic socio-technical approach to risk evaluation, strengthening language around halting development for unmanageable risks, and expanding the range of considered safeguards. We believe these adjustments will further strengthen NIST’s crucial role in shaping responsible AI development practices.
Background and Context
The rapid advancement of AI foundation models has spurred novel industry-led risk mitigation strategies. Leading AI companies have voluntarily adopted frameworks like Responsible Scaling Policies and Preparedness Frameworks, outlining risk thresholds and mitigation strategies for increasingly capable AI systems. (Our response to NIST’s February RFI was largely an exploration of these policies, their benefits and drawbacks, and how they could be strengthened.)
Managing misuse risks in foundation models is of paramount importance given their broad applicability and potential for dual use. As these models become more powerful, they may inadvertently enable malicious actors to cause significant harm, including facilitating the development of weapons, enabling sophisticated cyber attacks, or generating harmful content. The challenge lies not only in identifying current risks but also in anticipating future threats that may emerge as AI capabilities expand.
NIST’s new guidance on “Managing Misuse Risk for Dual-Use Foundation Models” builds upon these industry initiatives, providing a more standardized and comprehensive approach to risk management. By focusing on objectives such as anticipating potential misuse, establishing clear risk thresholds, and implementing robust evaluation procedures, the guidance creates a framework that can be applied across the AI development ecosystem. This approach is crucial for ensuring that as AI technology advances, appropriate safeguards are in place to protect against potential misuse while still fostering innovation.
Strengths of the guidance
1. Comprehensive Documentation and Transparency
The guidance’s emphasis on thorough documentation and transparency represents a significant advancement in AI risk management. For every practice under every objective, the guidance indicates appropriate documentation; this approach is more thorough in advancing transparency than any comparable guidance to date. The creation of a paper trail for decision-making and risk evaluation is crucial for both internal governance and potential external audits.
The push for transparency extends to collaboration with external stakeholders. For instance, practice 6.4 recommends providing “safe harbors for third-party safety research,” including publishing “a clear vulnerability disclosure policy for model safety issues.” This openness to external scrutiny and feedback is essential for building trust and fostering collaborative problem-solving in AI safety. (FAS has published a legislative proposal calling for enshrining “safe harbor” protections for AI researchers into law.)
2. Lifecycle Approach to Risk Management
The guidance excels in its holistic approach to risk management, covering the entire lifecycle of foundation models from pre-development assessment through to post-deployment monitoring. This comprehensive approach is evident in the structure of the document itself, which follows a logical progression from anticipating risks (Objective 1) through to responding to misuse after deployment (Objective 6).
The guidance demonstrates a proactive stance by recommending risk assessment before model development. Practice 1.3 suggests to “Estimate the model’s capabilities of concern before it is developed…”, which helps anticipate and mitigate potential harms before they materialize. The framework for red team evaluations (Practice 4.2) is particularly robust, recommending independent external experts and suggesting ways to compensate for gaps between red teams and real threat actors. The guidance also emphasizes the importance of ongoing risk assessment. Practice 3.2 recommends to “Periodically revisit estimates of misuse risk stemming from model theft…” This acknowledgment of the dynamic nature of AI risks encourages continuous vigilance.
3. Strong Stance on Model Security and Risk Tolerance
The guidance takes a firm stance on model security and risk tolerance, particularly in Objective 3. It unequivocally states that models relying on confidentiality for misuse risk management should only be developed when theft risk is sufficiently mitigated. This emphasizes the critical importance of security in AI development, including considerations for insider threats (Practice 3.1).
The guidance also demonstrates a realistic approach to the challenges posed by different deployment strategies. In Practice 5.1, it notes, “For example, allowing fine-tuning via API can significantly limit options to prevent jailbreaking and sharing the model’s weights can significantly limit options to monitor for misuse (Practice 6.1) and respond to instances of misuse (Practice 6.2).” This candid discussion of the limitations of safety interventions for open weight foundation models is crucial for fostering realistic risk assessments.
Additionally, the guidance promotes a conservative approach to risk management. Practice 5.3 recommends to “Consider leaving a margin of safety between the estimated level of risk at the point of deployment and the organization’s risk tolerance.” It further suggests considering “a larger margin of safety to manage risks that are more severe or less certain.” This approach provides an extra layer of protection against unforeseen risks or rapid capability advancements, which is crucial given the uncertainties inherent in AI development.
These elements collectively demonstrate NIST’s commitment to promoting realistic and robust risk management practices that prioritize safety and security in AI development and deployment. However, while the NIST guidance demonstrates several important strengths, there are areas where it could be further improved to enhance its effectiveness in managing misuse risks for dual-use foundation models.
Areas for improvement
1. Need for a More Comprehensive Socio-technical Approach to Measuring Misuse Risk
Objective 4 of the guidance demonstrates a commendable effort to incorporate elements of a socio-technical approach in measuring misuse risk. The guidance recognizes the importance of considering both technical and social factors, emphasizes the use of red teams to assess potential misuse scenarios, and acknowledges the need to consider different levels of access and various threat actors. Furthermore, it highlights the importance of avoiding harm during the measurement process, which is crucial in a socio-technical framework.
However, the guidance falls short in fully embracing a comprehensive socio-technical perspective. While it touches on the importance of external experts, it does not sufficiently emphasize the value of diverse perspectives, particularly from individuals with lived experiences relevant to specific risk scenarios. The guidance also lacks a structured approach to exploring the full range of potential misuse scenarios across different contexts and risk areas. Finally, the guidance does not mention measuring absolute versus marginal risks (ie., how much total misuse risk a model poses in a specific context versus how much marginal risk it poses compared to existing tools). These gaps limit the effectiveness of the proposed risk measurement approach in capturing the full complexity of AI system interactions with human users and broader societal contexts.
Specific recommendations for improving socio-technical approach
The NIST guidance in Practice 1.3 suggests estimating model capabilities by comparison to existing models, but provides little direction on how to conduct these comparisons effectively. To improve this, NIST could incorporate the concept of “available affordances.” This concept emphasizes that an AI system’s risk profile depends not just on its absolute capabilities, but also on the environmental resources and opportunities for affecting the world that are available to it.
Additionally, Kapoor et al. (2024) emphasize the importance of assessing the marginal risk of open foundation models compared to existing technologies or closed models. This approach aligns with a comprehensive socio-technical perspective by considering not just the absolute capabilities of AI systems, but also how they interact with existing technological and social contexts. For instance, when evaluating cybersecurity risks, they suggest considering both the potential for open models to automate vulnerability detection and the existing landscape of cybersecurity tools and practices. This marginal risk framework helps to contextualize the impact of open foundation models within broader socio-technical systems, providing a more nuanced understanding of their potential benefits and risks.
NIST could recommend that organizations assess both the absolute capabilities of their AI systems and the affordances available to them in potential deployment contexts. This approach would provide a more comprehensive view of potential risks than simply comparing models in isolation. For instance, the guidance could suggest evaluating how a system’s capabilities might change when given access to different interfaces, actuators, or information sources.
Similarly, Weidinger et al. (2023) argue that while quantitative benchmarks are important, they are insufficient for comprehensive safety evaluation. They suggest complementing quantitative measures with qualitative assessments, particularly at the human interaction and systemic impact layers. NIST could enhance its guidance by providing more specific recommendations for integrating qualitative evaluation methods alongside quantitative benchmarks.
NIST should acknowledge potential implementation challenges with a comprehensive socio-technical approach. Organizations may struggle to create benchmarks that accurately reflect real-world misuse scenarios, particularly given the rapid evolution of AI capabilities and threat landscapes. Maintaining up-to-date benchmarks in a fast-paced field presents another ongoing challenge. Additionally, organizations may face difficulties in translating quantitative assessments into actionable risk management strategies, especially when dealing with novel or complex risks. NIST could enhance the guidance by providing strategies for navigating these challenges, such as suggesting collaborative industry efforts for benchmark development or offering frameworks for scalable testing approaches.
OpenAI‘s approach of using human participants to evaluate AI capabilities provides both a useful model for more comprehensive evaluation and an example of quantification challenges. While their evaluation attempted to quantify biological risk increase from AI access, they found that, as they put it, “Translating quantitative results into a meaningfully calibrated threshold for risk turns out to be difficult.” This underscores the need for more research on how to set meaningful thresholds and interpret quantitative results in the context of AI safety.
2. Inconsistencies in Risk Management Language
There are instances where the guidance uses varying levels of strength in its recommendations, particularly regarding when to halt or adjust development. For example, Practice 2.2 recommends to “Plan to adjust deployment or development strategies if misuse risks rise to unacceptable levels,” while Practice 3.2 uses stronger language, suggesting to “Adjust or halt further development until the risk of model theft is adequately managed.” This variation in language could lead to confusion and potentially weaker implementation of risk management strategies.
Furthermore, while the guidance emphasizes the importance of managing risks before deployment, it does not provide clear criteria for what constitutes “adequately managed” risk, particularly in the context of development rather than deployment. More consistent and specific language around these critical decision points would strengthen the guidance’s effectiveness in promoting responsible AI development.
Specific recommendations for strengthening language on halting development for unmanageable risks
To address the inconsistencies noted above, we suggest the following changes:
1. Standardize the language across the document to consistently use strong phrasing such as “Adjust or halt further development” when discussing responses to unacceptable levels of risk.
The current guidance uses varying levels of strength in its recommendations regarding development adjustments. For instance, Recommendation 4 of Practice 2.2 uses the phrase “Plan to adjust deployment or development strategies,” while Recommendation 3 of Practice 3.2 more strongly suggests to “Adjust or halt further development.” Consistent language would emphasize the critical nature of these decisions and reduce potential confusion or weak implementation of risk management strategies. This could be accomplished by changing the language of Practice 2.2, Recommendation 4 to “Plan to adjust or halt further development or deployment if misuse risks rise to unacceptable levels before adequate security and safeguards are available to manage risk.”
The need for stronger language regarding halting development is reflected both in NIST’s other work and in commitments that many frontier AI developers have publicly agreed to. For instance, the NIST AI Risk Management Framework, section 1.2.3 (Risk Prioritization), suggests: “In some cases where an AI system presents the highest risk – where negative impacts are imminent, severe harms are actually occurring, or catastrophic risks are present – development and deployment should cease in a safe manner until risks can be sufficiently mitigated.” Further, the AI Seoul Summit frontier AI safety commitments explicitly state that organizations should “set out explicit processes they intend to follow if their model or system poses risks that meet or exceed the pre-defined thresholds.” Importantly, these commitments go on to specify that “In the extreme, organisations commit not to develop or deploy a model or system at all, if mitigations cannot be applied to keep risks below the thresholds.”
2. Add to the list of transparency documentation for Practice 2.2 the following: “A decision-making framework for determining when risks have become truly unmanageable, considering factors like the severity of potential harm, the likelihood of the risk materializing, and the feasibility of mitigation strategies.”
While the current guidance emphasizes the importance of managing risks before deployment (e.g., in Practice 5.3), it does not provide clear criteria for what constitutes “adequately managed” risk, particularly in the context of development rather than deployment. A decision-making framework would provide clearer guidance on when to take the serious step of halting development. This addition would help prevent situations where development continues despite unacceptable risks due to a lack of clear stopping criteria. This recommendation aligns with the approach suggested by Alaga and Schuett (2023) in their paper on coordinated pausing, where they emphasize the need for clear thresholds and decision criteria to determine when AI development should be halted due to unacceptable risks.
3. Gaps in Model Development Safeguards
The guidance’s treatment of safeguards, particularly those related to model development, lacks sufficient detail to be practically useful. This is most evident in Appendix B, which lists example safeguards. While this appendix is a valuable addition, the safeguards related to model training (“Improve the model’s training”) are notably lacking in detail compared to the safeguards around model security and detecting misuse.
While the guidance covers many aspects of risk management comprehensively, especially model security, it does not provide enough specific recommendations for technical approaches to building safer models during the development phase. This gap could limit the practical utility of the guidance for AI developers seeking to implement safety measures from the earliest stages of model creation.
Specific recommendations for additional safeguards for model development
For some safeguards, we recommend that the misuse risk guidance explicitly reference relevant sections of NIST 600-1, the Generative Artificial Intelligence Profile. Specifically, the GAI profile offers more comprehensive guidance on data-related and monitoring safeguards. For instance, the profile emphasizes documenting training data curation policies (MP-4.1-004) and establishing policies for data collection, retention, and quality (MP-4.1-005), which are crucial for managing misuse risk from the earliest stages of development. Additionally, the profile suggests implementing real-time monitoring processes for analyzing generated content performance and trustworthiness characteristics (MG-3.2-006), which could significantly enhance ongoing risk management during development. These references to the GAI Profile on model development safeguards could take the form of an additional item in Appendix B, or be incorporated into the relevant sections earlier in the guidance.
Beyond pointing to the model development safeguards included in the GAI Profile, we also recommend expanding Appendix B to include further safeguards for the model development phase. Both the GAI Profile and the current misuse risk guidance lack specific recommendations for two key model development safeguards: iterative safety testing throughout development and staged development/release processes. Below are two proposed additions to Appendix B:
The proposed safeguard “Implement iterative safety testing throughout development” addresses the current guidance’s limited detail on model training and development safeguards. This approach aligns with Barrett, et al.’s AI Risk-Management Standards Profile for General-Purpose AI Systems and Foundation Models (the “GPAIS Profile”)’s emphasis on proactive and ongoing risk assessment. Specifically, the Profile recommends identifying “GPAIS impacts…and risks (including potential uses, misuses, and abuses), starting from an early AI lifecycle stage and repeatedly through new lifecycle phases or as new information becomes available” (Barrett et al., 2023, p. 19). The GPAIS Profile further suggests that for larger models, developers should “analyze, customize, reanalyze, customize differently, etc., then deploy and monitor” (Barrett et al., 2023, p. 19), where “analyze” encompasses probing, stress testing, and red teaming. This iterative safety testing would integrate safety considerations throughout development, aligning with the guidance’s emphasis on proactive risk management and anticipating potential misuse risk.
Similarly, the proposed safeguard “Establish a staged development and release process” addresses a significant gap in the current guidance. While Practice 5.1 discusses pre-deployment risk assessment, it lacks a structured approach to incrementally increasing model capabilities or access. Solaiman et al. (2023) propose a “gradient of release” framework for generative AI, a phased approach to model deployment that allows for iterative risk assessment and mitigation. This aligns with the guidance’s emphasis on ongoing risk management and could enhance the ‘margin of safety’ concept in Practice 5.3. Implementing such a staged process would introduce multiple risk assessment checkpoints throughout development and deployment, potentially improving safety outcomes.
Conclusion
NIST’s guidance on “Managing Misuse Risk for Dual-Use Foundation Models” represents a significant step forward in establishing robust practices for mitigating catastrophic risks associated with advanced AI systems. The document’s emphasis on comprehensive risk assessment, transparent decision-making, and proactive safeguards demonstrates a commendable commitment to responsible AI development. However, to more robustly contribute to risk mitigation, the guidance must evolve to address key challenges, including a stronger approach to measuring misuse risk, consistent language on halting development, and more detailed model development safeguards.
As the science of AI risk assessment advances, this guidance should be recursively updated to address emerging risks and incorporate new best practices. While voluntary guidance is crucial, it is important to recognize that it cannot replace the need for robust policy and regulation. A combination of industry best practices, government oversight, and international cooperation will be necessary to ensure the responsible development of high-risk AI systems.
We appreciate the opportunity to provide input on this important document. FAS stands ready to continue assisting NIST in refining and implementing this guidance, as well as in developing further resources for responsible AI development. We believe that close collaboration between government agencies, industry leaders, and civil society organizations is key to realizing the benefits of AI while effectively mitigating its most serious risks.