Attempts to predict the occurrence or the likelihood of extreme acts of terrorist violence on the scale of 9/11 should be discouraged because the available data are too sparse to permit the reliable modeling of such “rare events,” according to a new report to the Pentagon (pdf) from the JASON defense advisory panel.
In a nutshell, “it is simply not possible to validate (evaluate) predictive models of rare events that have not occurred, and unvalidated models cannot be relied upon.”
On the other hand, the JASONs said, it may be possible and useful to assume that rare events are correlated with more frequent, observable events which can be reliably modeled. If one assumes that “rare events events occur on a continuum with more frequent events,” then the latter can be used to help predict the former.
In this way, the JASONs calculated that the probability of another 9/11-scale event in the world could be about 7% in the next ten years. But for reasons they went on to enumerate, the underlying assumption of continuity between rare and frequent events is not demonstrably correct.
“Much of the work on [anticipating] rare terrorist events seems to take for granted that ‘the truth is out there’ and we can discover it in a sufficiently timely fashion with the right mixture of motivational assessment, social network analysis, capability measures, etc.” This may not be true, they indicated.
The JASONs offered suggestions for improving the modeling process, and they stressed the need for “good, large datasets of [terrorist] events and incident data” that currently do not exist or are not widely available. It is “surprisingly hard to obtain primary datasets” even on “straightforward” questions of terrorist event frequency and magnitude.
They cautioned that the complexity of the problem and the presumed urgency of the threat have “led some to advocate the suspension of normal standards of scientific hypothesis testing, in order to press [predictive] models quickly into operational service.” But “while appreciating the urgency, JASON believes such advice to be misguided…. Experience in the development of many other scientific fields shows the importance of adhering to rigorous scientific standards, so that small successes are tested, communicated, critically examined, reproduced, and built upon.”
“Although patient husbandry of a long-term research program may fall short of addressing the immediate operational needs, JASON believes it is the best way forward for success in the long term.” A copy of the new JASON report was obtained by Secrecy News. See “Rare Events,” October 2009.
A lack of sustained federal funding, deteriorating research infrastructure and networks, restrictive immigration policies, and waning international collaboration are driving this erosion into a full-scale “American Brain Drain.”
With 2000 nuclear weapons on alert, far more powerful than the first bomb tested in the Jornada Del Muerto during the Trinity Test 80 years ago, our world has been fundamentally altered.
As the United States continues nuclear modernization on all legs of its nuclear triad through the creation of new variants of warheads, missiles, and delivery platforms, examining the effects of nuclear weapons production on the public is ever more pressing.
“The first rule of government transformation is: there are a lot of rules. And there should be-ish. But we don’t need to wait for permission to rewrite them. Let’s go fix and build some things and show how it’s done.”