Develop a Risk Assessment Framework for AI Integration into Nuclear Weapons Command, Control, and Communications Systems
As the United States overhauls nearly every element of its strategic nuclear forces, artificial intelligence is set to play a larger role—initially in early‑warning sensors and decision‑support tools, and likely in other mission areas. Improved detection could strengthen deterrence, but only if accompanying hazards—automation bias, model hallucinations, exploitable software vulnerabilities, and the risk of eroding assured second‑strike capability—are well managed.
To ensure responsible AI integration, the Office of the Assistant Secretary of Defense for Nuclear Deterrence, Chemical, and Biological Defense Policy and Programs (OASD (ND-CBD)), the U.S. Strategic Command (STRATCOM), the Defense Advanced Research Projects Agency (DARPA), the Office of the Undersecretary of Defense for Policy (OUSD(P)), and the National Nuclear Security Administration (NNSA), should jointly develop a standardized AI risk-assessment framework guidance document, with implementation led by the Department of Defense’s Chief Digital and Artificial Intelligence Office (CDAO) and STRATCOM. Furthermore, DARPA and CDAO should join the Nuclear Weapons Council to ensure AI-related risks are systematically evaluated alongside traditional nuclear modernization decisions.
Challenge and Opportunity
The United States is replacing or modernizing nearly every component of its strategic nuclear forces, estimated to cost at least $1.7 trillion over the next 30 years. This includes its:
- Intercontinental ballistic missiles (ICBMs)
- Ballistic missile submarines and their submarine-launched ballistic missiles (SLBMs)
- Strategic bombers, cruise missiles, and gravity bombs
- Nuclear warhead production and plutonium pit fabrication facilities
Simultaneously, artificial intelligence (AI) capabilities are rapidly advancing and being applied across the national security enterprise, including nuclear weapons stockpile stewardship and some components of command, control, and communications (NC3) systems, which encompass early warning, decision-making, and force deployment components.
The NNSA, responsible for stockpile stewardship, is increasingly integrating AI into its work. This includes using AI for advanced modeling and simulation of nuclear warheads. For example, by creating a digital twin of existing weapons systems to analyze aging and performance issues, as well as using AI to accelerate the lifecycle of nuclear weapons development. Furthermore, NNSA is leading some aspects of the safety testing and systematic evaluations of frontier AI models on behalf of the U.S. government, with a specific focus on assessing nuclear and radiological risk.
Within the NC3 architecture, a complex “system of systems” with over 200 components, simpler forms of AI are already being used in areas including early‑warning sensors, and may be applied to decision‑support tools and other subsystems as confidence and capability grow. General Anthony J. Cotton—who leads STRATCOM, the combatant command that directs America’s global nuclear forces and their command‑and‑control network—told a 2024 conference that STRATCOM is “exploring all possible technologies, techniques, and methods” to modernize NC3. Advanced AI and data‑analytics tools, he said, can sharpen decision‑making, fuse nuclear and conventional operations, speed data‑sharing with allies, and thus strengthen deterrence. General Cotton added that research must also map the cascading risks, emergent behaviors, and unintended pathways that AI could introduce into nuclear decision processes.
Thus, from stockpile stewardship to NC3 systems, AI is likely to be integrated across multiple nuclear capabilities, some potentially stabilizing, others potentially highly destabilizing. For example, on the stabilizing effects, AI could enhance early warning systems by processing large volumes of satellite, radar, and other signals intelligence, thus providing more time to decision-makers. On the destabilizing side, the ability for AI to detect or track other countries’ nuclear forces could be destabilizing, triggering an expansionary arms race if countries doubt the credibility of their second-strike capability. Furthermore, countries may misinterpret each other’s nuclear deterrence doctrines or have no means of verification of human control of their nuclear weapons.
While several public research reports have been conducted on how AI integration into NC3 could upset the balance of strategic stability, less research has focused on the fundamental challenges with AI systems themselves that must be accounted for in any risk framework. Per the National Institute of Standards and Technology’s (NIST) AI Risk Management Framework, several fundamental AI challenges at a technical level must be accounted for in the integration of AI into stockpile stewardship and NC3.
Not all AI applications within the nuclear enterprise carry the same level of risk. For example, using AI to model warhead aging in stockpile stewardship is largely internal to the Department of Energy (DOE) and involves less operational risk. Despite lower risk, there is still potential for an insufficiently secure model to lead to leaked technical data about nuclear weapons.
However, integrating AI into decision support systems or early warning functions within NC3 introduces significantly higher stakes. These systems require time-sensitive, high-consequence judgments, and AI integration in this context raises serious concerns about issues including confabulations, human-AI interactions, and information security:
- Confabulations: A phenomenon in which generative AI systems (GAI) systems generate and confidently present erroneous or false content in response to user inputs, or
prompts. These phenomena are colloquially also referred to as “hallucinations” or “fabrications”, and could have particularly dangerous consequences in high-stakes settings.
- Human-AI Interactions: Due to the complexity and human-like nature of GAI technology, humans may over-rely on GAI systems or may unjustifiably perceive GAI content to be of higher quality than that produced by other sources. This phenomenon is an example of automation bias or excessive deference to automated systems. This deference can lead to a shift from a human making the final decision (“human in the loop”), to a human merely observing AI generated decisions (“human on the loop”). Automation bias therefore risks exacerbating other risks of GAI systems as it can lead to humans maintaining insufficient oversight.
- Information Security: AI expands the cyberattack surface of NC3. Poisoned AI training data and tampered code can embed backdoors, and, once deployed, prompt‑injection or adversarial examples can hijack AI decision tools, distort early‑warning analytics, or leak secret data. The opacity of large AI models can let these exploits spread unnoticed, and as models become more complex, they will be harder to debug.
This is not an exhaustive list of issues with AI systems, however it highlights several key areas that must be managed. A risk framework must account for these distinctions and apply stricter oversight where system failure could have direct consequences for escalation or deterrence credibility. Without such a framework, it will be challenging to harness the benefits AI has to offer.
Plan of Action
Recommendation 1. OASD (ND-CBD), STRATCOM, DARPA, OUSD(P), and NNSA, should develop a standardized risk assessment framework guidance document to evaluate the integration of artificial intelligence into nuclear stockpile stewardship and NC3 systems.
This framework would enable systematic evaluation of risks, including confabulations, human-AI configuration, and information security, across modernization efforts. The framework could assess the extent to which an AI model is prone to confabulations, involving performance evaluations (or “benchmarking”) under a wide range of realistic conditions. While there are public measurements for confabulations, it is essential to evaluate AI systems on data relevant to the deployment circumstances, which could involve highly sensitive military information.
Additionally, the framework could assess human-AI configuration with specific focus on risks from automation bias and the degree of human oversight. For these tests, it is important to put the AI systems in contact with human operators in situations that are as close to real deployment as possible, for example when operators are tired, distracted, or under pressure.
Finally, the framework could include assessments of information security under extreme conditions. This should include simulating comprehensive adversarial attacks (or “red-teaming”) to understand how the AI system and its human operators behave when subject to a range of known attacks on AI systems.
NNSA should be included in this development due to their mission ownership of stockpile stewardship and nuclear safety, and leadership in advanced modeling and simulation capabilities. DARPA should be included due to its role as the cutting edge research and development agency, extensive experience in AI red-teaming, and understanding of the AI vulnerabilities landscape. STRATCOM must be included as the operational commander of NC3 systems, to ensure the framework accounts for real-word needs and escalation risks. OASD (ND-CBD) should be involved given the office’s responsibilities to oversee nuclear modernization and coordinate across the interagency. The OUSD (P) should be included to provide strategic oversight and ensure the risk assessment aligns with broader defense policy objectives and international commitments.
Recommendation 2. CDAO should implement the Risk Assessment Framework with STRATCOM
While NNSA, DARPA, OASD (ND-CBD) and STRATCOM can jointly create the risk assessment framework, CDAO and STRATCOM should serve as the implementation leads for utilizing the framework. Given that the CDAO is already responsible for AI assurance, testing and evaluation, and algorithmic oversight, they would be well-positioned to work with relevant stakeholders to support implementation of the technical assessment. STRATCOM would have the strongest understanding of operational contexts with which to apply the framework. NNSA and DARPA therefore could advise on technical underpinnings with regards to AI of the framework, while the CDAO would prioritize operational governance and compliance, ensuring that there are clear risk assessments completed and understood when considering integration of AI into nuclear-related defense systems.
Recommendation 3. DARPA and CDAO should join the Nuclear Weapons Council
Given their roles in the creation and implementation of the AI risk assessment framework, stakeholders from both DARPA and the CDAO should be incorporated into the Nuclear Weapons Council (NWC), either as full members or attendees to a subcommittee. As the NWC is the interagency body the DOE and the DoD responsible for sustaining and modernizing the U.S. nuclear deterrent, the NWC is responsible for endorsing military requirements, approving trade-offs, and ensuring alignment between DoD delivery systems and NNSA weapons.
As AI capabilities become increasingly embedded in nuclear weapons stewardship, NC3 systems, and broader force modernization, the NWC must be equipped to evaluate associated risks and technological implications. Currently, the NWC is composed of senior officials from the Department of Defense, the Joint Chiefs of Staff, and the Department of Energy, including the NNSA. While these entities bring deep domain expertise in nuclear policy, military operations, and weapons production, the Council lacks additional representation focused on AI.
DARPA’s inclusion would ensure that early-stage technology developments and red-teaming insights are considered upstream in decision-making. Likewise, CDAO’s presence would provide continuity in AI assurance, testing, and digital system governance across operational defense components. Their participation would enhance the Council’s ability to address new categories of risk, such as model confabulation, automation bias, and adversarial manipulation of AI systems, that are not traditionally covered by existing nuclear stakeholders. By incorporating DARPA and CDAO, the NWC would be better positioned to make informed decisions that reflect both traditional nuclear considerations and the rapidly evolving technological landscape that increasingly shapes them.
Conclusion
While AI is likely to be integrated into components of the U.S. nuclear enterprise, without a standardized initial approach to assessing and managing AI-specific risk, including confabulations, automation bias, and novel cybersecurity threats, this integration could undermine an effective deterrent. A risk assessment framework coordinated by OASD (ND-CBD), with STRATCOM, NNSA and DARPA, and implemented with support of the CDAO, could provide a starting point for NWC decisions and assessments of the alignment between DoD delivery system needs, the NNSA stockpile, and NC3 systems.
This memo was written by an AI Safety Policy Entrepreneurship Fellow over the course of a six-month, part-time program that supports individuals in advancing their policy ideas into practice. You can read more policy memos and learn about Policy Entrepreneurship Fellows here.
Yes, NWC subordinate organizations or subcommittees are not codified in Title 10 USC §179, so the NWC has the flexibility to create, merge, or abolish organizations and subcommittees as needed.
Section 1638 of the FY2025 National Defense Authorization Act established a Statement of Policy emphasizing that any use of AI in support of strategic deterrence should not compromise, “the principle of requiring positive human actions in execution of decisions by the President with respect to the employment of nuclear weapons.” However, as this memo describes, AI presents further challenges outside of solely keeping a human in the loop in terms of decision-making.
At this inflection point, the choice is not between speed and safety but between ungoverned acceleration and a calculated momentum that allows our strategic AI advantage to be both sustained and secured.
Improved detection could strengthen deterrence, but only if accompanying hazards—automation bias, model hallucinations, exploitable software vulnerabilities, and the risk of eroding assured second‑strike capability—are well managed.
A dedicated and properly resourced national entity is essential for supporting the development of safe, secure, and trustworthy AI to drive widespread adoption, by providing sustained, independent technical assessments and emergency coordination.
Congress should establish a new grant program, coordinated by the Cybersecurity and Infrastructure Security Agency, to assist state and local governments in addressing AI challenges.